A Favorite User Research Trick

by Jared M. Spool

“So, what do you think?”

That’s the question I posed to the executive stakeholder as we came out of an informative research session.

“It was about what I had expected,” the exec replied. How could that be? The participant in the session just exposed major challenges I knew none of us had considered before. How could that be something the executive expected.

This kept happening. We’d go into sessions that uncovered remarkable findings and the executives who observed would emerge claiming it was exactly as they expected. We listed all of the things we learned while the stakeholder struggled to acknowledge there was anything they didn’t expect to hear.

Asking for expectations up front

I made a change to my routine. While the observers were gathering before the next session, I described everything we knew about the participant we were about to meet. Then I asked each observer, “What do you think we’ll see in this session?” I walked through each of the research questions we’d plan to investigate and asked what they thought we’d learn from the participant.

I saved the executive for last. They had just heard the wide variety of guesses everyone was making and could now safely add their own to the list.

And guess what? The session didn’t go as anyone expected, even the executive.

When we emerged, I asked the exec what they’d thought of the session. “Wow, that was a huge surprise. It wasn’t at all what I had expected.” They went on to list all the things they had learned from the participant. Success!

Overcoming bad training

In many organizations, executives are trained to never show surprise. It’s part of the organization’s culture of pretending everything is always under control. Executives are supposed to be the ones in charge, they’re expected to know what the outcome will always be.

Of course, executives don’t always know. Nobody can always know. And the entire point of user research, whether it be user interviews, usability testing, field studies, or some other method, it to inform us of the things we couldn’t possibly know otherwise.

Every so often, someone gets it right!

I’m surprised when someone accurately predicts what we’ll learn. It’s happened a few times.

I keep an eye on that person. If in subsequent sessions, they continue to predict what we’ll learn from the participants, we’ll want to take advantage of their insights. They’re obviously in touch with the users in some magical way.

Having someone with this knowledge and experience on the team is truly valuable. We can trust them as a Subject Matter Expert. They’ll be a valuable resource.

Of course, we’ll continue to ask them to make predictions, to learn where the boundaries of their knowledge and experience extend. Everyone has their limits.

Part of our regular research protocol now

I now ask “What do you think we’ll see in this session?” before every session I conduct. And I rarely get ”it was about what I expected” as an answer because we have documentation showing it wasn’t at all what they expected.

I’ve found this to be a useful technique for helping every observer head into each research ready to experience a broader understanding of who the users are and what they need. This is key if we’re to deliver better-designed products and services.

Published here on April 12, 2019.