How a Team Matures Its User Research Integration
April 10, 2019
The order came down from high up (or wherever orders come down from): We need to implement an export data function in the app. That’s all we knew. Except there was a rumor of a big deal being held up until someone got their export data function.
The stage of our team’s user research maturity will determine how we’ll handle the user research for the export data function request. The more mature a team’s integration is, they more data and insights they have to make smart design decisions.
Stage 0: No user research integration at all
Starting out, our team hasn’t built up any user research capability yet. The developers go ahead and implement an export data feature based on what they think the requirements are, making guesses for anything not specified. They design something the way they themselves would like to use it. (A technique we call self design.)
It isn’t until the feature is delivered and users start using it that we get any feedback at all. What feedback we get likely comes from support calls or sales engineers, reporting back what they’re hearing from the customer.
The feedback is likely distorted and incomplete, so any recommendations or fixes that come from it aren’t effective. The most likely scenario is the feature stays in its initial design until some future feature requires a change.
Stage 1: Basic usability testing
Our team now conducts usability tests on new features. Once the developers give us something to put in front of folks, we gather up several users and ask them to give it a try.
When the participants show up, we jump right into a session. We give them some tasks to perform, based on what we imagine how our users will employ the export data function.
As they try to export the fake data we’ve created for them, we capture all the places where the interface confuses them or prevents them from accomplishing the task. When we’re done, we write a report.
Like Stage 0, our developers first took a crack at building what they thought would be a good response to the requirements. We used what they built for our test. Our testing sessions tell them if they were close or not.
Of course, we need to wait until there’s code written and stable enough to test. And there’s a good chance that there’s a hard deadline to ship the feature soon. That means we have to rush to get all the testing done that we’d like.
When the developers’ initial design didn’t work well for the users, there’s a big discussion about whether we ship what we’ve got anyways. (We do.)
Stage 1.5: Usability test tasks come from support
Our team starts working closely with the support and customer success teams. We hear directly from them about what our users are trying to do with the export data function. More importantly, we hear where it’s not working well.
As we conduct more usability tests on the functionality, we use this information to help us craft our tasks. We focus our research on problems users contacted support about, to ensure we better meet their expectations.
Stage 2: Interview-based task design
We upped our usability testing game. Instead of crafting test tasks based on how we imagine users will use the export data function, we use interview-based tasks instead.
Now, when recruiting participants, we screen them to ensure they’re likely to use the export data function in our design. Upon their arrival, we start the usability testing session with an interview. During that interview, we probe to understand how they might use the export data function. We work with them to jointly create a test task, which we then ask them to complete.
Even though no two users will use the same tasks, we learn much more about the export data function and what our users are trying to do. If we can do this testing early enough in the feature’s development, we can influence a better fit for the design.
Stage 3: Basic field research
Time to experiment with getting out of the building and heading to where the customers are. We are not trading in our usability testing program for this new field study work. That’s still happening. By doing both, we make our insights richer by better understanding the contexts around features like export data.
We’ve broken through the corporate politics that try to separate us from real users in their own environment. Now we can see where and how our users work with our design.
When we arrive for a field study, our research participants show us what they’re currently up to. Because the export data function is not something our users frequently need, the chances are slim that we’ll see them use it. However, we may see many upstream or downstream activities.
Researching in the “wild” lets us see more of what our users’ lives and work environments are like. We can feed this into the tasks and environments we design for our continuing laboratory studies. Even though our exposure to the export data function usage is limited, we’re learning a ton about our users overall.
Stage 4: Focused field research
We’ve upgraded our field research know-how to identify and recruit those field study participants who will use the export data function when we visit. Now we see what leads up to exporting the data, and what they use the exported data for.
Because we can focus in on the right users at the right moment, we can see a broader sample of issues that arise. We’re seeing more subtlety and nuance in how people use our design than we’ve ever seen before. In turn, this helps us identify what works best for our users.
Stage 5: Longitudinal studies
We now continually study the end-to-end experience of our users. We don’t wait to be told from above that there’s a need for the export data function. We know about it before they tell us. Our research told us there was a need before the customers mentioned it to the sales folks.
Our research techniques have become more nuanced. We improve our ethnographic methods with some deep hanging out with our users and customers. We experiment with storytelling listening techniques, such as collective story harvesting and World Café. We exercise our data science skills, deriving meaning from by crafting custom telemetry from our designs.
All this advanced research informs our entire team, feeding insights to prioritize their work. It provides a shared understanding so other team members are clear on the intention of the design. The better they understand the design’s intention, the more likely they’ll create something that matches it.
Stage 6: Strategic user research
Up until this point, our research efforts primarily focused on the current functionality. Yet, with our deep knowledge and understanding of the full user experience, we now play a strategic role throughout the organization.
Our research has become instrumental in picking strategic priorities for the product as a whole. The ethnographic field techniques are showing us where we can create a competitive advantage, not just from holes in our own experience, but from gaps competitors haven’t identified either. These strategic insights become the basis of the themes on our product’s roadmap.
Our research also feeds into the overall vision of a product’s experience, helping us understand exactly what work is left for our teams to do.
It’s team maturity, not organization maturity
These are the individual stages each of our organization’s teams go through. Each team goes through this process at their own rate.
Even if you’re a researcher that’s assigned to multiple teams, you’ve probably noticed your teams each have a different pace of increasing the user research integration into their work. Reducing friction for growing their research capability is the best way to help them speed it up. For example, you could build a centralized capability for recruiting, making it easier for teams to select participants for studies on short notice. (Studies are always on shorter notice than we’d like.)
Because every team is at a different stage of maturity, there’s no real advantage to assess the overall organization’s maturity. It’s better to track the progress of individual teams and identify what might nudge them to the next stage.
Beyond technical skills
At every new stage, our initial focus is on the new technical skills we must master to be good at our jobs. Yet, it doesn’t stop when we’ve mastered certain techniques.
We also must learn how to make our research valuable to the teams we work with. They need to feel our research is invaluable to their work.
We want to discover what piece of research will be so beneficial that our teammates will demand more. We need to learn what will make them so hungry for our research, they feel they can’t live without it.
Research can’t be a separate function
Outsourcing your research is like outsourcing your vacation. It gets the activity done, but it doesn’t realize the value. If only your researchers are doing the research, you’re doing it wrong.
For research to be truly valuable, everyone on the team has to be part of it. Everyone should be a part of the planning and have a say in who our participants will be and what we want to learn from them.
Every member of the team needs to observe the research and be part of the synthesis of the outcomes. They need to feel ownership in understanding the problems users have and potential solutions.
We can’t achieve this if we’re the intermediary between the rest of the team and the users. They need direct and constant exposure to the people they’re designing and developing for.
We need to teach them how to truly see their users and learn how to make great design decisions as a result. We’ll know we’ve succeeded when the team comes to better design recommendations than we would ever have made ourselves.
Researchers need to be research leaders
For the team’s user research integration to mature, a major role for the team’s researchers is to constantly be growing the team’s research capabilities. It’s our job to be thinking about how we’ll get to the next stage, while, simultaneously, we focus on how to better work at the current stage. We can’t skip a stage. Each is necessary a step to the next one.
We need to champion a culture of continuous learning—not only for ourselves—but for our teammates too. Every day, we need to celebrate what we’ve learned about our users, those users’ needs and challenges, and the techniques that helped us uncover them.
Much of a successful research practice is building relationships. Relationships with the gatekeepers to the customers, the customer success team, the customers, and their users. We need to earn their trust and respect their own needs. This, after all, are the key traits of a great leader.
User research is about growing and learning. Every day, we get better at what we do and our teams learn more about the people they’re designing for. This is how our organization will deliver better-designed products and services.
Solid User Research is a critically important capability to become a design-led organization. In our 2-day, intensive Creating a UX Strategy Playbook workshop, I’ll work directly with you and your team leaders to craft your ideal strategy to ensure your teams have the best UX research skills and experience.
We cap each workshop off at about 24 attendees. This gives Jared plenty of time to work directly with you on the ideal strategy for your team. Learn how your team can deliver better-designed products and services.