Lesson from user tests: shut up and listen
Jul 22 2013
Never before have I seen research, conjecture and real life come together so succinctly.
This week we sat down with a total of 24 Spanish speakers to determine what they thought about our mobile product. We did this in the form of a user test, a procedure that helps identify existing problems with product design and usability.
In this test, participants are asked to interact with our prototype and give their feedback. We asked users to tell us what they thought about our web application that provides a Spanish-language audio translation of a local English-language news broadcast on a smartphone. We wanted to know if users liked this product and would use it. We also were testing to see what exactly our users wanted and needed from a local news perspective and whether our product fulfilled that need.
We worked with three separate focus groups of 5-10 people whose characteristics matched those of our target market. The participants were beginning ESL students, and their primary language was Spanish. Their ages ranged from upper 20s to early 50s, and about half of them owned mobile phones.
We encountered one of our first challenges with the prototype as soon as we introduced it. Many of our users had trouble inputting the web address into a browser on their smartphones. Users typed the web address into a search engine and got related search results, not our prototype.
Once we got that issue sorted out, users started the application. A cacophony of Spanish voices erupted. We hadn’t realized that users should have simultaneously started the audio. Oops. But they adjusted by simply leaning in closer or turning the volume up as they watched the visuals on the English broadcast video.
We got to the meat of testing sandwich once the users finished listening to the Spanish version of the broadcast. We asked a variety of questions, but what stuck out to me as the most important was the first one on the list: What did you think of this product?
Now, let me backtrack for a minute. I’ve had my fair share of experience with interviewing and have led a research project but never has the importance of asking an open-ended question and shutting up been so apparent to me. It struck me during that first test that because the users are the ones with the need, they’re the ones who can tell us just exactly what they need. If I didn’t listen carefully, there was no way I was going to know.
The most important thing was to just put a question out there, sit back and listen without interruption. I learned much more by allowing awkward silences to grow and lengthen than from accepting an initial response and continuing on with questioning. Sometimes then, I found, those who don’t normally speak up will offer their opinion when you weren’t expecting it. And those responses are the nuggets, the golden gems that you’ve been hoping to get all along.
The user testing was a complete success. Our participants’ feedback not only corroborated the conclusions we have drawn from our own research, but they also confirmed that the need we had identified was real, and filled by our product. Users especially highlighted their desire to know what was going on in their local area and the scarcity of news sources that offered that information.
This week showed us that our product is something our users need, want and would definitely use. We had to field questions about when the application would be available for download, which of course made us happy. But it also made us realize how much further we’d need to take this product to make it a concrete reality. But for the moment, I’m happy with the lesson that I took away from our first user tests – shut up and listen.