In anticipation of the Advancing Research conference (in person, in New York, coming up in March), I spoke with Lou Rosenfeld about:
the state of the user research industry – where we’ve been, where we are, and where we’re headed. If the field of research was once a lonely desert, today it’s a jungle. It was once a field where researchers could get lost and forgotten. Today, the field is teaming with life—so much so that you could get eaten alive.
Gleaning lessons from the past, Steve doesn’t want us to forget the desert. But he has no desire to return there.
In his chat with Lou, they look back, and they look ahead. They discuss shifts in community and networking, and how research agencies are being replaced by in-house research teams. Finally, the two discuss Steve’s role in the upcoming, in-person Advancing Research conference in Queens, New York.
You can find our 40-minute conversation at the episode page and embedded below, twice.
Our conversation is in two parts (between 35 and 40 minutes each). You can find the audio and transcript at the the episode pages (Part 1, Part 2) and each are embedded below.
In connection with the second edition of Interviewing Users, I spoke with Brian O’Neill for his Experiencing Data podcast, titled “No Time for That:” Enabling Effective Data Product UX Research in Product-Immature Organizations.
“If you don’t know what you’re doing, and you don’t know what you should be investing effort-wise, that’s the inexperience in the approach. If you don’t know how to plan, what should we be trying to solve in this research? What are we trying to learn? What are we going to do with it in the organization? Who should we be talking to? How do we find them? What do we ask them? And then a really good one: how do we make sense of that information so that it has impact that we can take away?”
“What do people get [from user research]? I think the chance for a team to align around something that comes in from the outside.”
On the impact user research can have if teams embrace it: “They had a product that did a thing that no one [understood], and they had to change the product, but also change how they talked about it, change how they built it, and change how they packaged it. And that was a really dramatic turnaround. And it came out of our research, but [mostly] because they really leaned into making use of this stuff.”
“If we knew all the questions to ask, we would just write a survey, right? It’s a lower time commitment from the participant to do that. But we’re trying to get at what we don’t know that we don’t know. For some of us, that’s fun!”
It was a real treat to speak with Matt Valle for the Rock n’ Roll Research Podcast.
Steve Portigal of Portigal Consulting has been doing User Research since the days our software all came in a box. He has written a seminal book on the topic, “Interviewing Users: How to Uncover Compelling Insights,” recently released in a 2nd edition available through Rosenfeld Media.
Steve shares his journey from the nascent years of user research through today and his take on where user research is headed. We discuss his book and how studying creative writing has informed his approach. Steve also tells the story of building a community of Rolling Stones enthusiasts – pre-World Wide Web! – that is still alive and kicking (just like Keith Richards).
You can find our 35-minute conversation on the YouTube and embedded below.
Episode #105: Steve Portigal - User Research Expert, Author, Rolling Stones Enthusiast
Thanks to Chad McAllister and Product Mastery Now for their interview with me, titled How product managers best interview users (bonus: it’s still good advice if you aren’t a product manager)
You can find our 35-minute conversation on the episode page (and on Google) and embedded below.
471: How product managers best interview users – with Steve Portigal
Summary
[2:52] Why did your book, Interviewing Users, need a second edition? It’s been 10 years since the first edition was published. The fields that we all work in have changed. There was a little bit of discussion 10 years ago about remote user research, and now remote research is much more common. I wanted to talk in-depth about the best practices for remote research, even as they’re still emerging. Research operations, which is a field adjacent to user research, has emerged. The book also draws from 10 more years of me doing research and teaching research. I’m always learning. I updated the stories and included better examples.
[6:09] How do we ask customers the right questions? First, don’t assume you know what people want. Second, recognize that just asking customers what they want is not effective. There are a few related questions that you should answer.
Business challenge: What do we want to do? What do we want to change? What’s coming up? Why are we doing this research? Research question: What do we want to learn from people? Interview questions: The questions you ask customers.
What you want to learn is not the same as what you should ask. For example, if you want to understand where people find the most value in their budgetary spending, don’t ask, “Where do you find the most value in your budgetary spending?” Instead, craft a set of questions and build a discussion guide that has a flow and sets context. Ask questions like:
What do you do?
How do you do it?
How long have you been doing it?
What are you big problems?
Where does budgeting fit into those larger problems?
Use the interview to ask many questions to get a larger context so you can conclude what the answers to your research questions are.
[10:19] How should we prepare for a customer interview? Once you understand your business question and research question, think about your sample. Who are you going to talk to? Be creative in your sample. Don’t talk to the same people over and over again. Be intentional about who is going to give you the most information. Talk to people who will give deeper insight about the situation so you can make decisions about the changes you want to make. Figure out who will give you answers to your research question.
Next, figure out how to get to those people.
Then figure out what you’re going to ask them. Write a discussion guide. No interview looks like the guide you write, but it is a great tool to share with stakeholders to respond to their questions. This is a case of “Chance favors the prepared mind.”
[12:42] How do you avoid interviewing the wrong sample? It’s much better to have a stakeholder object to your sample at the beginning of the study than at the end. Have a rationalization for the people you are choosing to talk to. Make everyone aware of the tradeoffs and how far these interviews will take you in your understanding.
[15:13] What do you do during the interview? Don’t go through the interview line by line. All the good stuff comes from follow-ups. In the best interview, you ask one question and everything else follows. Every question you ask comes from something they just said. You are connecting and engaging such that everything they say is important because you have another thing you want know that builds on it. Telling someone honestly that their information is important and valuable to you creates rapport that makes these interview successful. Asking follow-up questions is a great rapport builder. You’re trying to poke around in the dark with a flashlight to see what’s behind the corners. You’re looking for your own understanding, not just checking off questions. Follow-up questions come from your own curiosity.
You can’t really do an interview by starting with a single question. There are points where you have to switch gears. That technique is really important. I call it signaling your lane changes. Tell someone, “Well, this is really great. I want to switch topics a little bit here and now move on to your procurement process.” Again, this tells them that what they’re saying is important.
[5:35] How do we stay curious and avoid developing biases? Cognitive bias is very natural, and I don’t want anyone to feel bad about it because I think feeling bad about it makes it harder to overcome. When I do an interview and uncover one of my biases, that’s the most fun feeling. You could easily feel stupid because you have biases, but I think we’re doing this research to learn things so it’s exciting when someone knocks our sandcastle down. Not every bias you overcome is an insight about your product, but it puts you in that mode where you realize you’ve been holding onto something. Be able to hear when you have biases and feel good about that.
The skill is not to not have biases. The skill is to be able to hold on to multiple truths at once. Set your worldview aside and use the interview to embrace somebody else’s worldview. Set the intention that makes you curious.
You want to hear how they approach their problem or your product. That doesn’t diminish what you know. Being curious and having a beginner’s mind does not negate your expert’s mind. You just compartmentalize them for a time during the interview.
[24:42] How do you ask questions? I like to have a first interviewer and a second interviewer. The first interviewer controls the flow of the interview. The second interviewer records the interview, listens deeply, and identifies things to be curious about that have gotten past the first interviewer. Before transitioning to a new topic, the first interviewer asks the second interviewer if they have any questions.
[27:35] How do you analyze data from interviews? In general, you might do two hours of analysis for every one hour of synthesis. Analysis is taking large things and breaking them down into smaller ones. You can pull out some things from an interview about the person you’re interviewing. These are not conclusions, just distillations.
Synthesis is taking small things and organizing them in a new way to make something larger. You reorganize the broken-down pieces from many interviews. This can be done with affinity maps. You start creating frameworks that create segmentation or a list of priorities.
How much analysis and synthesis you do depends on the size of your project. Don’t just tabulate what people said—you’ll miss all the nuance. You’re creating your narrative, a new story that’s put together from the data.
I was on the Greenbook podcast recently, in a episode titled Beyond the Surface: Navigating the Depths of User Research with Steve Portigal.
Check out our 40-minute discussion on the episode page (and on Spotify, Apple, and Google) and embedded below in two different formats.
Excerpt:
There’s this interesting part of research where it’s collaborative and facilitative…I can do a better job if I can help them learn something and take something away. But, if I hear what they’re taking away, especially [as] I’m not the domain expert. I work as a consultant, so I come into an area that somebody else inhabits. And so they’re going to always see things in the research that I won’t see. It’s really helpful for me to understand what didn’t they hear that person say. Like, if there’s a gap in what they took away, then I now know I need to kind of emphasize that because there’s a takeaway that’s obvious to me that isn’t to them. So I can get that out a debrief. And, when I hear what they heard and what surprises them, I understand, yeah, how they’re framing the world, what’s relevant information. I’m getting this indirect feedback.
There was a semi viral post about how to go into a bar and show people stuff. I think it was meant to be helpful, but it was kind of a smart-ass post…”give people beer and ask them a question.”
It seems like, if one is trying to help somebody else learn a skill or utilize a process or a practice – I guess everything is on a continuum. But you can see one end of the continuum says “this is really hard” and the other end of the continuum says “this is really easy.” And I mean, hopefully I’m somewhere in the middle. I think I’m trying to say this is hard, but here’s how to get there. If that’s my story, I feel pretty critical of people that are like, oh, anyone can do this. This is easy.. There are books and posts like this that kind of say that.
And I think there’s something to be said for lowering barriers, reducing intimidation, kind of giving people some power and some confidence. So we got to ask ourselves, like, is that person, you know, all exhausted from talking to one person in the office? Is it better that they did that than they just sat in their cube and kind of thought how smart they were? Is the team going into the bar or the one person going, is that better or not than not doing that? And I think argument for “better” is like any effort you make to get out of your own head is, it can be well intentioned.
And the evil part of me hopes that people try and fail –if you go to a bar and think if you just ask somebody one question in a bar about something that’s on your laptop and you fail somehow in that and you at least learn that you failed, you know, now I feel like now I’m going to hand you my book and say like, okay, you know, here’s how to get there.
And so maybe people need to try these things where someone says “Oh, yeah, it’s super easy just to ask this one question.”
That’s not how I would go about it. I would like to say “This is hard for the following reasons and here’s the way to get there.”
All This ChittahChattah is the Portigal blog, bringing you observations of daily life, disdainful snark over bad design, updates on my activities, and info about user research since 2001.
