When users attack: What to do when user testing doesn’t go your way

Mar 24 2014

We scroll through the user responses, feeling almost smug as we check off the answers.

Would you use this product? Yes.

Would you use it more than twice a week? Yes.

Would you read on your tablet? Yes.

Would you pay $15 or more annually for a subscription? No.

No? No more than $15? This has to be wrong. We wanted them to say $25. We needed them to say at least $20.

We pause, looking up from the screen. Around the computer, brows furrow and eyes cloud over with worry. This is the first time in the process of testing our product that we’ve been faced with, not quite a rejection, but at least setback.

Since pitching the idea for OpEd magazine, a digital, opinion-based magazine, we’ve moved through the incubation period seamlessly. We’ve created an efficient group dynamic, perfected our elevator pitch and developed a working prototype. But now that we’re in what’s supposed to be the most exciting part of the process—user testing—we’re stumped.

We look back at the screen, calculating and evaluating. Finally one of us speaks up.

Maybe we add new features to raise the anticipated value?

Maybe we could release stories more often so readers become more dependent?

Maybe we could deal with having lower subscription rates and aim for more advertising revenue instead?

Suddenly we’re back. And, in that minute-long period of uncertainty, we realized something: we needed that. We entered the process of user testing assuming that it would be a big pat on the back, the legitimate means of confirming all of our assumptions. But that’s not how it works.

Because user testing is supposed to challenge our assumptions. It’s supposed to allow us to see beyond the biases that color our relationship with the product, even if that sometimes means rethinking what has once seemed obvious.

And though this isn’t easy, it’s necessary. And it brings us one step closer to the product we want to create, whether or not it was the one we started with.

Editor’s note: Cheney is working on OpEd Magazine. OpEd Magazine is a digital, opinion-based magazine that changes the way readers access intelligent and interesting commentary on trending issues. It pits insightful and witty contributors against one another to cover sports, entertainment, politics, business, lifestyle and pop culture.


Leave a Reply

Your email address will not be published. Required fields are marked *