There were notions about very basic yet crucial product features that were overturned for us. Better now than after weve written the code for it. Sematores, Inc.
We heard all kinds of things, some were validating earlier suspicions, while others were things we completely missed. The more data points the better. Tripzon
The workshop was great, this is exactly what we were looking for and need right now. -Jaze
They [Fidelity] have this framework that makes it really easy to process and categorize user feedback so we can actually do something about it. Buyjacker
Theres certainly an art to asking these questions Id love to learn how to do this better so that next time I can do it myself. AET
As Eric Ries says in The Lean Startup (Im going to paraphrase) If you dont think your product is ugly then youve probably waited too long to get something into the hands of the user. Traditional early phase user-testing entailed surveys, challenging the user to think theoretically but taking this approach has multiple pitfalls:
1) in general users have an easier time reacting to something tactile or visual in a constructive way
2) furthermore if your product is a disruptive technology users will not be able to imagine the need for a product without something they can react to
3) users are typically more comfortable delivering positive feedback rather than negative feedback because negative feedback requires the support of detailed examples
Bottomline: Do not be afraid to seek feedback early and often. While receiving negative feedback can be difficult, you would rather hear it early rather than when your resources are completely drained, which would make pivoting impossible. Your prototype doesnt need to be perfect just get it out there! Eric Ries is a proponent of what he calls a minimally viable product; this is a barebones prototype of your product which includes all of its critical features, usually in a rudimentary fashion, that users can nonetheless react to. Do not be afraid to put something scrappy in front of your users!
Q: So then what are the guidelines for user testing?
A: There are many schools of thought when it comes to user testing, where some approaches are more appropriate than others depending on where you are in development. Nevertheless its always healthy to go into a test session with several hypotheses but whatever you do - do not get married to these ideas. Fidelity coaches encouraged Finalists to do little more than give the user minimum context, e.g. Tripzons setup was Imagine you need to plan a triphere use this. Thats it.
From then on out Finalists were allowed to ask questions akin to those your therapist might ask you How are you feeling? What are you thinking? Why is that important? What would make you use this over an existing product? But absolutely under no circumstances, should you pitch or defend a feature or try to justify anything. Try it - its not easy. The next step is synthesis which entails prioritizing feedback and evaluating what can be completed reasonably in the upcoming 2-3 weeks that will have the maximum impact possible on the products development.