Listen to Steve on The Universal Lens Channel

an overhead view of rapidly moving traffic and highway infrastructure with the title the Universal Lens Channel

Thanks to Chris Kovel for having me on the inaugural episode of his “In Dialogue” series on the Universal Lens Channel. For about 75 minutes, we talked about the state of user research in 2023.

The episode is on YouTube but the interview is audio only. The YouTube and audio versions are embedded below, and also available on YouTube and libsyn.

E1: In Dialogue with Steve Portigal

There’s a transcript available on YouTube.

You don’t have visibility into everything and so I think [a company’s user research maturity] needs sometimes a dedicated examination and consideration in order to to improve it. I guess that’s how I would make the case or because you know at a profession level there are things that we can do but ultimately the implementation is taking place inside companies and what it really looks like is very localized. Company A and Company B can learn from each other in terms of what their best practices are, what their struggles are, but it’s hyper local — we have this way of doing product management, we have this market, we have this maturity in our marketing business, we have this kind of product, this vertical — all those things are going to really change what it takes to build a more mature practice, and if you don’t locally examine it, and you know, what more mature looks like for company A is not what more mature looks like for Company B. So I think there is sort of an investment needed of time and focus and ideally an external perspective to try to see where strides can be made to to to yeah to move things to move things along.

Second edition of Interviewing Users now available for pre-order

It’s been 10 years since I wrote Interviewing Users, and I’m thrilled to announce the second edition! It comes out October 17th and is available for pre-order, at a 15% discount.

In this new and updated edition of the acclaimed classic Interviewing Users, Steve Portigal quickly and effectively dispels the myth that interviewing is trivial. He shows how research studies and logistics can be used to determine concrete goals for a business and takes the reader on a detailed journey into the specifics of interviewing techniques, best practices, fieldwork, documentation, and how to make sense of uncovered data. Then Steve takes the process even further—showing the methods and details behind asking questions—from the words themselves to the interviewer’s actions and how they influence an interview. There is even a chapter on making sure that information gleaned from the research study is used by the business in such a way to make it impactful and worthwhile. Oh, and for good measure he throws in information about Research Operations.

Who Should Read This Book?

  • Anyone and everyone who is interested in finding out what makes their business tick, i.e., who their users are.
  • Anyone and everyone who wants to learn how to interview and listen to people.
  • Anyone and everyone, including CEOs, user researchers, designers, engineers, marketers, product managers, strategists, interviewers, and you.

Bonus: read Chapter 1 here.

Steve contributed to “Remote User Research: 12 Tips from Experts”

There’s a really good range of ideas and best practices in Remote User Research: 12 Tips from Experts. Mine is below

2. Set expectations for your participants
If you’re carrying out remote research via video conferencing tools, it’s always a good idea to set expectations before the interview.

“It’s likely that a remote user research session, however you understand that term, is going to be something different for your participant,” explains experienced user researcher Steve Portigal. “It’s not a work meeting over Zoom, it’s not catching up with friends over Facetime. So begin your session by calling attention to anything that might be especially awkward for either of you.”

Steve suggests pointing out that you might not be making eye contact, for example. Let the participant know that you’re going to be taking notes while they talk and that—even though you won’t look at them directly—you’ll be listening to them and watching what they’re doing.

“Eye contact works at a human perceptual level,” Steve stresses. “So it’s not clear that you can simply explain away something about how the brain works. But, at worst, it serves to establish the rapport and frame this session as a collaborative endeavor.”

3. Consider not using video or only doing so sparingly
We’re currently seeing a lot of articles about “Zoom fatigue”, which argue that the slight glitching of video, and the expectations for posture and gaze, are not sustainable for our mental health. So while it may feel counter-intuitive, audio may actually be preferable for making a connection with another person.

“Terry Gross, the host of NPR’s talk show Fresh Air, for example, is famous for eschewing any visual context at all for her interviews,” Steve points out. “Similarly, the psychotherapist’s traditional couch serves to create intimacy specifically by avoiding looking at the other person.”

Steve therefore suggests beginning a remote session with video to greet each other, but then to keep it off for the conversational parts of the interview. Experiment and see what works for you—and for your participants.

Here’s a whole pile of new episodes of Dollars To Donuts

Dollars to Donuts

In the past couple of months I’ve managed to post a raft of new episode of Dollars To Donuts, the podcast where I speak with people who lead user research in their organization.

Here are the latest episodes

You can also find the podcast on Stitcher, Spotify, Google Play, and Apple.

Steve Portigal on Reusable Research, Interview War Stories, and Letting Go of Implicit Beliefs

I did a brief Q&A with Bloomfire. The original is here and I’ve reposted it below.


As just about any market research professional would be quick to tell you, their job isn’t just about conducting research: it’s about disseminating what they learn to the stakeholders who need that information to make informed decisions. As a result, successful market researchers spend a lot of time thinking about how to share their knowledge in the most impactful way possible.

It seems fitting then, that the second entry in our Future of Knowledge at Work series features Steve Portigal, a researcher (as well as an author and podcaster) who has built his career on helping businesses better understand the people who use their products or services.

Tell us a little about yourself.
I’m a user researcher, and I’ve run my own consulting practice since 2001. I lead user research efforts for teams, and I also serve as a coach for teams doing their own research, and I also teach organizations how to improve their research skills. I live in a small, foggy coastal town just outside of San Francisco where there are lots of trails for walking my dog.

How did you get started in your career as a user researcher, and what led you to start your own consultancy?
After graduate school I ended up at a small consultancy that was experimenting with user research as a service that could bring value to their clients, and I was able to get on board in those early days. I have always liked consulting for the variety and how that provides continuous learning opportunities about almost everything I do. So when that agency failed to make it through the stock market crash, it seemed like the obvious next step to go out on my own.

What knowledge do you rely on most to do your job?
As a user researcher, I’m usually working with knowledge from two different groups— the team that makes something (or wants to make something), and the people that the team hopes will buy or use that thing. My job is to uncover and synthesize new knowledge about both parties (e.g, how is this artifact or service understood, how is it being used, and how might it be used?) and then work to resolve the gaps between them.

In your experience working on research projects for your clients, what do you think are the biggest knowledge management challenges (and how do you solve them)?
There’s often a certain amount of anxiety about the data (say, recordings and transcripts of user research sessions), but the knowledge is the key to having the research be impactful and lead to meaningful change. But that change can be painful, as it means letting go of previously held beliefs. It’s especially difficult with those that are implicit beliefs.

To have the most impact, I share anecdotes and examples along the way— not to present patterns and conclusions too early, but to introduce the rest of the team to the real people whose stories are the basis of the work that we’re doing, and then I work collaboratively with them to take apart and reassemble all that data into knowledge. Being part of that process— which is messy and divergent at points— really makes them the owners of the new knowledge that we’re creating. And then, we work to transform knowledge into (potential) actions.

Thoughts on how new technologies will impact the way researchers manage and share their results?
This is such a hot topic among growing research teams. The need is evolving from sharing results and archiving, to really building up a collection of data and knowledge that can be revisited. Siloed research and repetitive research are inefficient and hurt the cause of bringing research into every part of the organization, so the more the work of researchers (from data through results) can be made accessible and reusable, the more impact research can have inside the organization.

What best practices would you recommend for capturing and sharing knowledge?
When I work with people who are new to doing research, I urge them to record their user research interview sessions. People think that they’re pretty good note-takers, but for research, you need to capture exactly what people say, and you can’t take notes quickly enough to do that. In the moment is not the time to be editing and interpreting.

If you record the research, then you have a high-fidelity record that can be transcribed and then can be both analyzed (e.g, pulling out key quotes, examples, anecdotes) and synthesized (e.g., arranged into new themes, frameworks, opportunities). It’s crucial that people have a good model for how to treat the knowledge that they are building in this process and that they understand when and how to emphasize the customer’s point of view, and when and how to insert their own interpretive point of view.

Can you tell us a little about your book Doorbells, Danger, and Dead Batteries?

I wrote a book called Interviewing Users that is a pretty solid introduction to (go figure) interviewing users. I’ve long been interested in the fascinating experiences that researchers have going into people’s lives, physically going into homes and workplaces and other environments. Sometimes things go wrong in amazing ways. We have encounters that are hilarious, embarrassing, distressing, confounding, and more. These become stories that researchers share informally, but I hadn’t ever seen anyone collect these stories, let alone reflect on how these stories can be used to teach us about the practice of research.

As part of writing Interviewing Users, I set up an online channel for documenting these war stories. Now, as much as we create structure and rigor and tools around the work of research, there’s an element we can’t control—we’re meeting other people in their space. But that’s the beauty of research: these experiences teach us and change us. Doorbells, Danger, and Dead Batteries collects the stories of 65 researchers from around the world and unpacks the lessons they provide.

Can you tell us a bit about your podcast, Dollars to Donuts, and what inspired you to start it?
Dollars to Donuts features interviews with people who are leading in-house research teams. As companies have invested more in user research, there’s now more people in research-specific leadership roles. This is something that didn’t exist until recently. I wanted to discover and share their stories and best practices as an extension to the consulting work I do with many different teams.

How Louis Theroux asks questions

From 2016, here’s a great video by Ryan Hollinger analyzing how documentary filmmaker asks questions, builds rapport, uncovers information. This video doesn’t get into one of my personal favorites of Louis’s superpowers – that he ignores negative energy – discomfort, anger, superiority – from his interviewees. Rather than fleeing, he stays calm and continues to engage.

(update: video link corrected)

How Louis Theroux Asks A Question | Ryan's Theory

Making an Impact with UX Research Insights

I wrote a short piece for UX Mastery about ensuring the results of user interviews have the most impact. I’ve included an excerpt below, but you should read the whole piece here.

In your analysis, how do you decide what’s important?

Take time at the beginning of the research to frame the problem. Where did this research initiate? What hypotheses – often implicit ones – do stakeholders have? What business decisions will be made as a result of this research?

What research reveals doesn’t always fit into the structure that is handed to you ahead of time, so knowing what those expectations are can help you with both analysis and communication. Some things are important to understand because they’re part of the brief. But other things are going to emerge as important because as you spend time with your conclusions you realise “Oh this is the thing!”

David Isay on selective memory

Krista Tippett interviewed David Isay for her show On Being. He talked about interviewing his father and his story highlights the gap between what we remember from an interview and what actually transpired in that interview.

DI: I remember that I asked him when we were in the StoryCorps interview, “What are you proudest of in life?” And my memory of that was that he said “the books I’ve written.” And I always teased him. I said, “Dad, we’ve done, whatever, 10,000, 20,000, as time went on, 50,000 interviews, and everybody says their kids. And you, the one person, you said, ‘my books.’” And I just endlessly went after him, and the night he died, I listened to the interview, and I said, “What are you proudest of?” And he said “My kids.”

KT: Really?

DI: Yep.

KT: Was that exchange even in there? What you remember? You just didn’t remember it?

DI. ISAY: Yes, and then he said, “I’m also proud of my books.”

Takeaway: Record your interviews and go back to those recordings — don’t rely on your memory!

A “first interview” story

Jennifer Kim talks about her experience in preparing for (or not) conducting her first interviews. She is honest about her mistakes, and what she’s learned. I found myself feeling critical of her general neediness: when a participant doesn’t react well to her unprepared interviewing, she is hurt; when a participant gives her feedback and encourages her, she takes that to heart. It’s her job to make the participant feel good, not the other way around. But that lesson may come later, she’s the rawest of beginners and is revealing her own vulnerability in the experience, and I give her full credit for that strength of character.

RSA Interaction Design

(thanks to Christina Wodtke)

My talk about “How to Interview Users to Uncover Insights”

I recently spoke about How to Interview Users to Uncover Insights the Lean Product Meetup in Silicon Valley. They’ve just posted the video and I’ve embedded it and the slides below.

Steve Portigal on How to Interview Users to Uncover Insights at Lean Product Meetup

It’s a wrap for Dollars to Donuts, Season 2

I just wrapped up the second season of Dollars to Donuts, the podcast where I speak with people who lead user research. Check out all the great interviews this season. Links include transcripts and links for each episode.

An interview about The State of UX Research

I was interviewed by Jen Ignacz of Topp. We spoke about the history of user research (at least how I experienced) and some of my thoughts about the present – and future. Check out the audio and/or read the transcript here.

I remember that we did this project with IBM that was very much like the future of the home PC, so for us that was really, really new and exciting. Maybe a lot of people might be rolling their eyes like yes, we’ve seen that we’ve done that, so that was this watershed moment where we were able to do a sort of an industrial design type of project, but it led with ethnography – it led with rethinking the whole purpose of this thing they were making. And right after that we got approached by a packaged goods companies that wanted to rethink breakfast, and that was the exciting part because their innovation part of the business was getting clients that didn’t look like industrial design clients. It was someone else coming through the door, and that was the moment where I think we thought “this is a real thing” – you know, companies – business is looking into this and we can work on all kinds of stuff. I think that was a huge moment. Fortune, BusinessWeek and other magazines were writing cover stories about ethnography or anthropology, and showing pictures of people in pith helmets or scientists or similar. The conversation turned a lot more serious and specific about how this kind of work was going to help business. I think the work we were getting and we were doing, and this kind of popular press shift, we started to feel like oh, this really is a viable thing for business, a viable service to be offering. We will see products made this way from here on out, so that was kind of the transition.

Grant McCracken on his interview technique and mindset

Another fantastic Grant McCracken post. He conducts a short interview (embedded below) and offers a terrifically insightful reflection on his technique as well the meaning of the overall endeavor. A must-watch/read for interviewers.

Craig Young in the street in SF

Another thing I liked about the interview was the glimpse it gives of city life. In this case, of the invisible distinctions of space that are perfectly clear to Craig and a surprise to the rest of us (if and when discovered by the rest of us). The world is filled with this invisible distinctions. They surround us all the time. The secret of ethnography: keep an eye out. Ask everyone.

How To Tell If Your Participant Is Faking It

Although How To Tell If Your Participant Is Faking It is mostly about usability testing and unfortunately chooses to frame the participant as “faking” (a nice word for lying) it’s nice to see this level of specific detail around the clues to look for in terms of how people express themselves.

Your participant reflects in the 3rd person. If a majority of the feedback your participant gives includes phrases such as “Some people might…” or “I have a friend who would love this…” or any other reference to someone other than themselves, then you’re probably not getting great data. They’re not exactly faking it or hiding anything, but they’re definitely not giving you relevant data about their thoughts, feelings, and behaviors. Be sure to clarify whether they mean to really speak for themselves or not.

Listen to Steve on the Getting2Alpha podcast

I was interviewed by Amy Jo Kim for her Getting2Aplpha podcast.

Those transition rituals tell people when they are going to go into an interview, there’s a point at which you’re getting ready, you’re going over there. You’re going from an activity that’s not the interview to an activity that is the interview. Just to take a moment and say, “Okay, for the next thirty minutes, the next hundred twenty minutes, let’s just try to learn about how Joanne makes travel reservations. Let’s just make it about that.” The implicit part of that statement is that we’re not thinking about what meetings we have, what aspirations we have, what sales targets we have to make, what our burn rate for our coders is. We’re really thinking about just looking at this person and their behavior.I think that’s just a way to give yourself a break and just make it easier, not easy, but easier to really think about that experience with that person and learning about them as a complete thing. Afterwards, you can leave and you can go back and pick up your world view and make sense of this and start to triangulate and organize and learn. For those periods of time where you’re with someone to learn about them, just taking that weight of the world off your shoulders and just saying, “Okay, I’m just going to learn about them.” The transition ritual is to consciously articulate that.

Listen to the podcast here or below

Whose Job is User Research?

I was interviewed by Laura Klein and we discussed Whose Job is User Research?

Not every research study requires an expert at the helm. Quite a few products would benefit from having somebody on the main product team who could quickly get feedback or answers to simple questions. “Even a newbie researcher should be able to answer some factual questions about what people are doing or might want to do. They also have the opportunity to reflect on what assumptions they were holding onto that were incorrect,” Steve explains. “You’ll always get more questions to go with your answers, but hoo boy–it’s better than never talking to users and acting with confident ignorance.”There are some questions you’re better off bringing in an expert, though. “The more help you need in connecting the business problem with the research approach and connecting the observations to the business implications, the more expert help you need,” Steve explains.

Check out the whole article.

Learning from kids interviewing bands

Kids Interview Bands is a series of videos of, obviously, kids interviewing bands. I propose these videos as a lightweight teaching tool for interviewing as it’s curious to see what goes well and what doesn’t.

The different kid interviewers are coming with a set of stock questions which they ask one after another, with practically no followup. So they never become an actual conversation and the amount of insightful revelation is low. But the kids are real, as little kids, and many of the band members respond to them in a real way.

I first came across this interview with Tom Araya of Slayer as an overall bad example, but I found it incredibly charming.

These people have very little in common, and perhaps limited skills in overcoming that gulf, but Araya never talks down to them, he never plays up his persona, he just does his best to connect with them, never forgetting they are children.

In a slightly different vein, I also liked this interview with Pustulus Maximus of GWAR who absolutely stays within the bounds of his horrific persona, but is kind and entertaining with the kids. He manages to work within his character and the context of the interview, and even though he plays a sort of monster, he doesn’t act like one.

In some ways, the limitations of these interviewers (they are just kids!) highlights other aspects that can contribute to a good interview – participants that take on some of the labor of establishing rapport and making that connection with the other person. And even if the kids don’t ask good follow-up questions (or any), their naturalness serves as an invitation to the musicians to meet them in that state.

A listening bootcamp

ListenUp_logo

Listen Up! is an online program from WNYC to help improve listening skills.

It’s not easy to improve our hearing. But everyone has the ability to become a better listener. Only Human invites you to participate in a listening bootcamp, with guidance from a memory champion, a world-class mediator, actors and improv comics, we’ve got five challenges designed to help you sharpen your listening skills.

An inflection point for user research: scandal

A user researcher is front and center in a Silicon Valley competitor-workers legal snarl.

One now-former employee, Ana Rosario, was hired by Fitbit as a user experience researcher around April 16 but did not disclose that she planned to leave Jawbone until April 22, the complaint said. On April 20, according to the complaint, Ms. Rosario held a meeting with Jawbone’s senior director of product management to discuss the company’s future plans and then downloaded what the company said was a “playbook” outlining its future products.During her exit interviews, Ms. Rosario initially denied taking confidential information, but she later acknowledged downloading its “Market Trends & Opportunities” presentation, the complaint said.

I don’t know Ana (although LinkedIn shows me I know many people who do know her); I do know people at Jawbone (and probably people at Fitbit). I’m not sharing this to comment on any of the players or the details of the situation itself, but to note at a meta-level that in the trajectory of user research as a business function, it’s grown in prominence to the point where you can pick up the New York Times one morning and see a story like this.

Interviewing Users: Link Roundup (Second Anniversary Edition)

interviewing-users

Wow. It’s been two years since my first book Interviewing Users was released. Here’s a roundup of links to various bits connected with the book. If you haven’t already, you should buy a copy here! It would be fantastic if you wrote a quick review on Amazon here.

The Book

Reviews

Interviews

Presentations

Other

Five Questions with Steve Portigal

This Friday I’ll be speaking at 18F in DC about The Power of Bad Ideas. The talk will be streamed here.

In advance of the talk, I answered a few questions about working with clients and planning research projects. Here’s a snippet; more at the 18F site.

SP: I’m intrigued by the user-centered theater — that is to say, people who have a design goal or a strategic need or a hunger for some insights, but who aren’t open to collaborating on how to accomplish that.

You often see this with projects where a client wants to understand something enormously complex and nuanced, and they don’t have any budget or time to do so. This is a big red flag. Sometimes, it’s worthwhile having a conversation to see if they [potential client] are open to feedback on their situation and on alternative ways to work.

In some cases, I’m pleasantly surprised; in many cases, though, I’m usually happy to pass on these projects. The kicker is that many of these folks have often already defined the method they want to use to reach their stated goal. It’s foolhardy to try to help people who have set you up to fail.

Video from Interviewing Users talk at HOW Interactive

I was finally able to get my hands on the video from November’s talk (Interviewing Users: Uncovering Compelling Insights) at the HOW Interactive Design conference. Below is the video both on YouTube and Vimeo (in case you have a preference?). Also below are the slides.

Interviewing Users: Uncovering Compelling Insights by Steve Portigal

Interviewing Users: Link Roundup

interviewing-users

Welcome to 2016! The New Year seems a good time to post my occasionally updated roundup of links to various bits connected with Interviewing Users. Of course, if you haven’t already, you should buy a copy here! It would be fantastic if you wrote a quick review on Amazon here.

The Book

Reviews

Interviews

Presentations

Other

The Advantages of Remote Interviewing

This NYT Magazine profile of author Laura Hillenbrand explores her writing process and considers the ways it has been impacted by her illness. One section of this excellent article had resonance for me around conducting remote interviews, something I’m frequently asked about.

One hallmark of literary nonfiction is its emphasis on personal observation. But Hillenbrand found that telephone interviews do offer certain advantages. No one appreciates this perspective more than the radio host Terry Gross, who performs nearly every interview on her program, “Fresh Air,” by remote. Gross told me that she began this habit, as Hillenbrand did, by necessity: The cost of bringing a guest to her studio in Philadelphia was simply too high. Over time, she said, she has come to believe that there is intimacy in distance. “I find it to be oddly distracting when the person is sitting across from me,” she said with a laugh. “It’s much easier to ask somebody a challenging question, or a difficult question, if you’re not looking the person in the eye.” Gross also said the remote interview makes it easier to steer the conversation. “I can look at my notes without fear that the interviewee will assume that I’m not paying attention to what they’re saying,” she said. Finally, the distance eliminates nonverbal cues, which can interfere with good quotes. “A hand gesture might be helpful to communicate something to me. It communicates nothing to my listeners.” Hillenbrand, who recorded hundreds of hours of interviews with Zamperini, experienced a similar effect. “I thought it was actually an advantage to be unable to go to Louie,” she said. Because neither of them had to dress for the interviews and they were in their own homes, their long phone calls enjoyed a warmth and comfort that might otherwise be missing. She could pose the deeply personal questions that even her father had trouble answering.

Designing the Problem, my keynote from ISA14 (now with video)

I gave a (remote) presentation, Designing the Problem, at Interaction South America a few weeks ago.

Too often we assume that doing research with users means checking in with them to get feedback on the solution we’ve already outlined. But the biggest value from research is in uncovering the crucial details of the problem that people have; the problem that we should be solving.

As the design practices mature within companies, they need to play an active role in driving the creation of new and innovative solutions to the real unmet needs that people have. In part, driving towards this maturity means looking at one’s own culture and realizing the value of being open-minded and curious, not simply confident. This is a challenge to each of us personally and as leaders within our teams and communities.

We’ve got video, slides, audio and sketch-notes. Enjoy!

The talk is just over 40 minutes and there are two questions (which you can’t hear but which should be obvious enough from my response).


To download the audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac)

Sketchnote by Kat Davis (click for full size)

Kat-Davis-B3KkgEsCcAE2Jjo

Sketchnote by Thiago Esser
10802698_389325447886104_12

Building Rapport With Users Is Building Rapport With People

We use specific techniques to build rapport in user research, but of course those techniques work to build rapport in other situations. Here’s two great examples of rapport in relationships that map quite closely to rapport in fieldwork.

Caution: Stuffed Shirts Ahead [NYT]

Instead, miss no opportunity to chat congenially with your new colleagues — lunch, coffee, the proverbial water cooler, whatever. But remember, these conversations aren’t about you. Though you don’t want to seem evasive, avoid leaping into a happy reminiscence about foosball tournaments with your delightful former colleagues.

Think of the process as the workplace equivalent of politicians’ “listening tours” during the run-up to election season. Don’t ask, “So what’s it like to work here?” or “Do you like it here?” or anything else that requires a point-blank value judgment. Ask neutral questions like, “So how long have you worked here?” Then keep quiet.

People love to talk about themselves, and if you can signal that you’re actually interested in what they’re saying — and not merely waiting for your turn to talk — most will do it all day long. (One of the oldest interview tricks that reporters use is silence: There’s a human tendency to fill a conversational void, so let the other person do it.) In addition to signaling that you’re going to fit in, you’ll likely pick up useful clues to help you do precisely that.

It may take a little patience, but you’ll gradually be able to piece together what you need to know about how this new environment works — and who among your new colleagues has the same kind of sensibility as yours. Remember that even if the company is formal and bureaucratic, chances are that at least some of the people who work there are, in fact, agreeable human beings — the kind who like doughnuts.

5 ways to build a good relationship with anyone [The Week]

I picked up a copy of an underground indie best-seller called It’s Not All about Me: The Top Ten Techniques for Building Quick Rapport with Anyone. The author, Robin Dreeke, is in charge of the Federal Bureau of Investigation’s elite Counterintelligence Behavioral Analysis Program. Robin combines hard science and years of work in the field to offer practical tips to build rapport and establish trust.

  1. Ask them questions.
  2. Don’t be a conversation dictator.
  3. Allow them to talk.
  4. Genuinely try to understand their thoughts and opinions.
  5. Leave your ego at the door.

Keeping the Humanity in our Technology Work

A few articles about the practice of medicine echo each other in significant ways, but I share them here as a reminder that all of our work that increasingly relies on technology (e.g., developing digital products) will suffer terribly if we fail to engage the human who thinks, talks, listens and tells stories.

With Electronic Medical Records, Doctors Read When They Should Talk

Even if all the redundant clinical information sitting on hospital servers everywhere were error-free, and even if excellent software made it all reasonably accessible, doctors and nurses still shouldn’t be spending their time reading. The first thing medical students learn is the value of a full history taken directly from the patient. The process takes them hours. Experience whittles that time down by a bit, but it always remains a substantial chunk that some feel is best devoted to more lucrative activities.

Enter various efficiency-promoting endeavors. One of the most durable has been the multipage health questionnaire for patients to complete on a clipboard before most outpatient visits. Why should the doctor expensively scribble down information when the patient can do a little free secretarial work instead? Alas, beware the doctor who does not review that questionnaire with you very carefully, taking an active interest in every little check mark. It turns out that the pathway into the medical brain, like most brains, is far more reliable when it runs from the hand than from the eye. Force the doctor to take notes, and the doctor will usually remember. Ask the doctor to read, and the doctor will scan, skip, elide, omit and often forget.

Like good police work, good medicine depends on deliberate, inefficient, plodding, expensive repetition. No system of data management will ever replace it.

Why Doctors Need Stories

I have long felt isolated in this position, embracing stories, which is why I warm to the possibility that the vignette is making a comeback. This summer, Oxford University Press began publishing a journal devoted to case reports. And this month, in an unusual move, the New England Journal of Medicine opened an issue with a case history involving a troubled mother, daughter and grandson. The contributors write: “Data are important, of course, but numbers sometimes imply an order to what is happening that can be misleading. Stories are better at capturing a different type of ‘big picture.’ ”

Beyond its roles as illustration, affirmation, hypothesis-builder and low-level guidance for practice, storytelling can act as a modest counterbalance to a straitened understanding of evidence. Thoughtful doctors consider data, accompanying narrative, plausibility and, yes, clinical anecdote in their decision making. To put the same matter differently, evidence-based medicine, properly enacted, is judgment-based medicine in which randomized trials, carefully assessed, are given their due.

I don’t think that psychiatry — or, again, medicine in general — need be apologetic about this state of affairs. Our substantial formal findings require integration. The danger is in pretending otherwise. It would be unfortunate if psychiatry moved fully — prematurely — to squeeze the art out of its science. And it would be unfortunate if we marginalized the case vignette. We need storytelling, to set us in the clinical moment, remind us of the variety of human experience and enrich our judgment.

From October 2003, Diagnosis Goes Low Tech

“This technology has become a religion within the medical community,” said Dr. Jerry Vannatta, former dean of the University of Oklahoma College of Medicine. “It is easy to lose sight of the fact that still, in the 21st century, it is believed that 80 to 85 percent of the diagnosis is in the patient’s story.”

Yet medical educators say that doctors are insufficiently trained to listen to those stories. After all, there is no reimbursement category on insurance forms for it. It is this lost art of listening to the patient that has been the inspiration behind a burgeoning movement in medical schools throughout the country: narrative medicine.

The idea that medical students need an academic discipline to teach them how to listen may strike some as farfetched. After all, what should be more natural — or uncomplicated — than having a conversation?

But the narrative medicine movement is part of an ongoing trend in exposing medical students to the humanities. It is needed, educators say, to teach aspiring doctors to pay close attention to what their patients are saying and to understand the way their own emotions affect their perceptions, and ultimately their clinical practice.

The basic teaching method is to have medical students read literary texts and then write about themselves and their patients in ordinary language, rather than in the technological lexicon of the traditional patient chart.

Venerable medical journals like The Journal of the American Medical Association and Annals of Internal Medicine are increasingly publishing reflective writing by doctors, their editors say. And now some medical schools even have their own literary journals. At Columbia University College of Physicians and Surgeons, there is Reflexions; Pennsylvania State University College of Medicine publishes Wild Onions; at the University of New Mexico’s Health Sciences Center, there is The Medical Muse.

Columbia also publishes a semiannual scholarly journal devoted solely to narrative medicine, titled Literature and Medicine, which is edited by Maura Spiegel, a literary scholar, and Dr. Rita Charon, a professor at the medical school and a founder of the narrative medicine movement.

Designing for Unmet Needs, my presentation from Warm Gun

Last week I spoke at the Warm Gun conference, giving a short talk about Designing for Unmet Needs

Don’t be surprised if Steve Portigal, author of Interviewing Users, invites himself to your family breakfast or follows hotel maintenance staff to the boiler room. For more than 15 years, he’s led hundreds of interviews that help clients understand customers and turn insights into design opportunities.

Steve knows that our success depends on letting the unmet needs of our audience shape our designs. Okay—but how do we hit a target we can’t see? How do we design for people who aren’t us? How do we solve for the complexity of those people?

Dig into the details, ditch the guesswork, and join Steve to engage deliberately with the people we’re designing for. Look at ways to acknowledge the complexity of your users. Offer solutions rooted in the connections you make with people. Get unstuck and discover opportunities for design that adds value.

Below you’ll find slides, audio and a sketchnote.

The talk is 25 minutes long.

To download the audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac)

Sketchnote by Lexi H (click for full size)

LEXI-H-B4Crtn5CcAA51R3

When your participant repels and scares you

Embedded above is a fantastic and disturbing episode of Love + Radio. Nick van der Kolk and Noah Morrison visit Jay Thunderbolt, who upon beginning the interview, aggressively reiterates his demand for payment (not possible for public radio, which Jay knew). Noah ends up going on a liquor run instead, as Jay offers him a pistol or Kevlar vest. Jay never stops insulting the interviewers, and stories of violence abound; indeed at one point he points a gun at Nick’s head. Meanwhile, they are interviewing Jay about the strip club he runs out of his house.

Yikes. This sounds like some of the War Stories, doesn’t it?

I don’t know what is going through Nick’s head as he’s doing this interview, but as I listen I find myself strongly repelled by Jay. And while the interview here is edited, so we don’t know all that happened, but Nick never reveals discomfort or lets Jay’s obvious provocations get to him. His patience and tolerance create room for Jay’s story to come out, and while Jay is not an appealing individual, you begin to understand and accept him as he is. Well, I did. Your experience may vary.

Nick finally responds to Jay’s taunting at the end, when he asks Jay “Do you think you understand the way I feel about you?” and Jay admits that he doesn’t. It’s a powerful moment in an intense interview.

Interviewing Best Practices from Stephen Colbert

The first episode of Working (a new podcast) features Stephen Colbert explaining in great detail the process of creating The Colbert Report. The entire episode (listen here) is really good process stuff (creativity, collaboration, finding the story, media firehose, working under pressure) but I want to call out the section about how he prepares and uses the questions for interviewing his guest, as it’s is quite consistent with what I wrote in Interviewing Users.

And then I read the two sheets of questions that the writers have come up, what their ideas are. I usually pick 10 or 15 of those. But I don’t look at them. I don’t look at them until right before I go over [to the set], and then I read them over once again in front of my producers to get a sense of, oh, this is how my character feels about this person.

Come show time…I take them out and I go, oh, yes, these are the questions I chose. And then I try to forget them and I try to never look at the cards. I just have a sense in my head of how I feel. And the cards are in front of me, but I try not to look at them at all. I’m pretty good. Maybe I look once a week at the cards. I put my hand on them, so I know I have them if something terrible happens, but as long as I know what my first question is for the guest I kind of know what every other question is, because I really want to react to what their reaction to my first question is.

And I usually end up using four of the 15, and the rest of it is, what is the person just saying to me? Which makes that the most enjoyable part of the show for me. Because I started off as an improviser. I’m not a standup. I didn’t start off as a writer, I learned to write through improvisation, and so that’s the part of the show that can most surprise me. The written part of the show, I know I can get wrong. You can’t really get the interview “wrong.”

Overcoming bias and developing empathy

Here are two interesting articles that feed right into the themes of my workshop, The Designer is Present, happening at the end of this month at UX Australia.

An Appeal to Our Inner Judge is about how biases – judgements we make quickly about others – are natural but can be overcome. The excerpt below comes at the end and is applicable to many things, not the least of which is becoming a better user researcher.

Recognize and accept that you have biases. Develop the capacity to observe yourself in action and to notice when certain people or circumstances serve as triggers.

Learning to slow down decision-making, especially when it affects other people, can help reduce the impact of bias. This can be particularly important when we are in circumstances that make us feel awkward or uncomfortable.

No Time to Think considers our always-on culture and the reluctance we have (as a result?) to be in the off position and (ulp!) alone with our thoughts. In the quoted part below, from the end of the article, it makes the case for what I’m aiming for with the workshop; that presence and mindfulness are essential for the work that many of us are doing.

Studies suggest that [a lack of presence] impairs your ability to empathize with others. “The more in touch with my own feelings and experiences, the richer and more accurate are my guesses of what passes through another person’s mind. Feeling what you feel is an ability that atrophies if you don’t use it.”

Contextual research from a bygone era

While listening to This American Life I learned about Roger Barker, a psych professor who turned the small Kansas town of Oskaloosa into a laboratory in the late 1940s.

Barker was one of the most extraordinary — and least known — figures in the history of psychology. Shortly after he became chair of the KU psychology department in the late 1940s, he relocated his family to Oskaloosa to observe and gather data about the residents who lived in the town, population 725.

At that time, psychological research was primarily done in laboratories. “It was the era of running rats through mazes to understand human behavior,” he said. “Barker said you won’t learn about any real human behavior in a laboratory. If psychologists want to understand human behavior in the real world, they must enter the real world.”

More from this article

Among Barker’s more unusual efforts was a 1951 paper he co-wrote under the title “One Boy’s Day.”

It chronicled 14 hours in the life of a local boy with the pseudonym Raymond Birch . He was 7 when Raymond’s parents allowed the Midwest Psychological Field Station to record his every movement, according to Sabar’s book:

7:00. Mrs. Birch said with pleasant casualness, ‘Raymond, wake up. …’
7:01. Raymond picked up a sock and began tugging and pulling it on his left foot. …
7:07. Raymond turned to his dresser and rummaged around among the things on it until he obtained a candy Easter egg for his dog.

The notations, archived at KU, track Raymond on his walk to school. He finds a baseball bat in the grass and swings it, accidentally striking a flagpole.

“This made a wonderful, hollow noise,” researchers wrote, “so he proceeded to hit the flagpole again.”

Barker eschewed academic prose and wanted his charges to record any telling, prosaic detail.

Through the 1950s, Oskaloosans grew accustomed to the sight of a child being shadowed by a note-scribbling adult. In published papers, this was the town of “Midwest,” in keeping with the scientific practice of shielding the identity of the subjects being examined.

Barker’s work differed from other scholarly studies of places such as Muncie, Ind., (Middletown) and Candor, N.Y., (Springdale) in at least two ways.

First, it focused less on class and politics and more on the relationships that made kids feel comfortable.

Second, Barker’s family settled into Oskaloosa as a permanent home. Roger and Louise continued to live there until their deaths, Roger’s in 1990 at age 87 and Louise’s in 2009 at 102.

While Barker used many methods, the part that struck me was his belief that simply documenting in exhaustive detail the ordinary activities throughout the day would somehow provide some additional insight. What would Barker have made of today’s era of personal analytics, data smog, quantified self and beyond?

Facing Your Fears: Approaching People For Research

intercept

I’ve always found intercepts – where researchers stop people on the street and ask them to participate in a quick study – to be challenging. (I also prefer to have longer interactions with people and even have them prepare for those research conversations, but that is a bit outside the point here). In Facing Your Fears: Approaching People For Research Carolyn Chandler breaks that challenge down into many pieces and addresses each of them. It’s a wonderful article because it gets deeply into the specifics and considers the mindset we bring to the activity and how to reframe that, in many different ways.

Rejection is people’s number one fear when approaching strangers. Hearing no has always been difficult, whether it’s a polite no or an angry no followed by a rant. Either way, it stings. Your response to that sting, though, is what matters. How do you explain the rejection to yourself, and does your explanation help or hurt you?

Martin Seligman, one of the originators of positive psychology, conducted a study in the ’70s that gives insight into the types of mindsets that make people feel helpless. Seligman found that those who exhibit long-term “learned helplessness” tend to view negative events as being personal, pervasive and permanent. In other words, if a person is rejected, they might rationalize that the rejection is a result of their own failing, that everyone else is likely to reject them as well, and that they can do nothing to lessen the likelihood of rejection.

When you prepare to approach someone, consider instead that, if they say no, they aren’t really rejecting you, but rather rejecting your request. It’s not personal. Maybe they’re in the middle of something, or maybe they’re just not in the mood to talk. The rejection is fleeting, and the next person might be perfectly happy to participate.

Young students do “fieldwork” to learn about others

In For Lessons About Class, a Field Trip Takes Students Home very young children are exposed to the homes and possessions of others. The thrust seems to be about class, but to me it seems like establishing an early model for empathy as well. The notion that other people are different from you seems foundational and it’s exciting to see this being addressed experientially. Check out the slideshow for the worksheets and debrief sessions!

Some of us have more toys and bigger homes than others. We all have a lot in common, but there are certain things that make us unique, too. Let’s talk about those things and celebrate them, even. This is not standard prekindergarten curricular fare, but it’s part of what the 4- and 5-year-olds at the Manhattan Country School learn by visiting one another’s homes during the school day. These are no mere play dates though; it’s more like Ethnography 101. Do classmates take the bus to school or walk? What neighborhood do they live in? What do they have in their homes? Over the last several weeks, I tagged along to find out. The progressive private school considers the visits to be one of the most radical things it does. “We knew we needed to talk about social class,” said Lois Gelernt, the teacher who came up with the idea. “It was opening up a can of worms, but if we were never going to talk about who we are and where we come from, the sense of community wasn’t going to be there.”

The personal commitment required for truly immersive research

Alice Goffman’s ‘On the Run’ Studies Policing in a Poor Urban Neighborhood [NYT] describes a sociologist who commits deeply to truly immersive fieldwork. At one level this simply reminds us of the differences between academic and industry work, but beyond that it surfaces just how personally demanding it is to deeply engage in a culture, requiring us to forgo much of ourselves (In Interviewing Users that’s Check Your Worldview At the Door) in order to understand the people we are interested in (Embrace How Other People See the World).

Ms. Goffman comes from a home where intensive fieldwork was something of a family business. Her father, the eminent sociologist Erving Goffman (who died when she was a baby), posed as an employee of a mental hospital for a year to research his 1961 study, “Asylums.” Her mother, Gillian Sankoff, is a sociolinguist at the University of Pennsylvania who has done studies in Papua New Guinea and French Canada; her adoptive father, the sociolinguist William Labov, also at Penn, has done pioneering field research on African-American urban vernacular, among other subjects.

Ms. Goffman, who grew up in the Center City neighborhood of Philadelphia, said she took her first field notes as a teenager, recording observations about the Italian-American side of her family in South Philadelphia. By her sophomore year at Penn, she had moved full time to a mixed-income African-American neighborhood and was hanging out on a tough strip, fully immersing herself in local culture.

She abandoned her vegetarian diet, listened only to mainstream hip-hop and R&B, and adopted local “male attitudes, dress, habits, and even language,” as she puts it in a long appendix, describing her research methods. While drugs, and drug selling, pervaded the neighborhood, she did not use them, she writes, partly because “it hampered writing the field notes.”

By her own account, she lost most of her college friends, and struggled to complete her non-sociology requirements.
It may sound “absurd” now, Ms. Goffman said of her extreme immersion. “But I was trying to take the participant-observer approach as seriously as possible.”

It can be hard to square the very ordinary-seeming academic who recalls her teenage affection for “My So-Called Life” with the young woman of her startlingly confessional appendix, which ends with a moving account of a close friend’s death in a shootout.

Listen to Steve on Wise Talk

Insights

I was interviewed by Sue Bethanis for Mariposa Leadership’s Wise Talk show. In an episode titled The Art of Interviewing Users we talked about how to see and notice in a different way, being aware of our own filters and biases, and constantly rediscovering what the problem you are trying to solve really is. You can listen to the interview below, or at Wise Talk.

To download the audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac).

One way to recruit in-home research participants

Credit card issuer Capital One isn’t shy about getting into customers’ faces. The company recently sent a contract update to cardholders that makes clear it can drop by any time it pleases. The update specifies that “we may contact you in any manner we choose” and that such contacts can include calls, emails, texts, faxes or a “personal visit.”

Yep, that’s how you get willing research participants – add it to the Terms of Service! Sure, I’m kidding; that’s not really what this article is about (it’s about the credit-card company claiming rights to repossession for non-payment). Still, this ups the creepy ante for visiting customers and makes the trust aspect of recruiting participants just that much harder.

Full story

Steve interviewed about Best Practices For Interviewing Your Audience

interviewing-users

Interviewing Users is now available. Get your copy here!

I spoke about Interviewing Users with Connie Malamed for the eLearning Coach podcast. Listen at the link, or below.

WE DISCUSS:

  • The impact that interviewing has on the product and the designer
  • The places in a design/development cycle where user input is needed
  • Skills needed to become good at interviewing users
  • What you’ll need to unlearn
  • The best attitude to have while interviewing
  • How to think about and manage “reticence”
  • Techniques to reinforce learning about the interviewee
  • How to plan for an interview
  • Tips for asking good questions
  • Ways to record interviews

Steve quoted in “Digital Products Flunking User Test”

I was interviewed by CruxialCIO about how companies can design and redesign digital and mobile products that engage rather than frustrate. The article (Digital Products Flunking User Test) is broken across three tabs (SituationSolutionsTakeaways) and the quotes from me appear on the second two tabs. FYI, the pages are a bit slow to load.

Remember the precedents. While copying competitors isn’t necessarily advisable, it doesn’t make sense to design, for example, a fly swatter that you use by swinging a string around with the flat swatter piece attached to it. People expect a stick at the end. “You can’t fail to acknowledge that there are precedents out there,” says Portigal.

“There’s some history about how customers are going to expect something to work. Everyone is a consumer so in an enterprise situation, we bring in expectations about how something should work.” If people expect that swiping left or right, double clicking, or other gestures will have a certain outcome, the lack of that outcome will be confusing. After Apple came out with the iPhone, for example, it became quickly clear that when consumers wanted a smartphone, they expected something fairly similar in form factor and function to the iPhone.

Grant McCracken’s brilliant “Ethnography, a brief description”

Eloquent awesomeness by Grant McCracken

The object of ethnography is to determine how the consumer sees the product, the service, the innovation. Often, this is obscure to us. We can’t see into the consumer’s (customer’s, viewer’s, user’s) head and heart because we are, in a sense, captive of our own heads and hearts. We have our way of seeing and experiencing the world. This becomes our barrier to entry. Ethnography is designed to give us a kind of helicopter experience. It takes up out of what we know and lowers us into the world of the consumer.
Ethnography is a messy method. In the beginning stages, we don’t know what we don’t know. We don’t know what we need to ask. We are walking around the consumer’s world looking for a way in. Eventually, as we ask a series of questions, we begin to see which ones work. We begin to collect the language and the logic the consumer uses. And eventually, we begin to see how they see the world.

The method is designed not to impose a set of questions and terms on the discussion, but to allow these to emerge over the course of the conversation. We are allowing the consumer to choose a path for the interview. We are endowing them with a sense that they are the expert. We are honoring the fact that they know and we don’t. (Because they do!)

Eventually, we end up with a great mass of data and it is now time to stop the ethnography and start the anthropology. Now we will use what we know about our culture, this industry, these consumers, this part of America to spot the essential patterns that make these data make sense. ”Slap your head” insights begin to emerge. ”Oh, that’s what their world looks like!” “That’w what they care about!” ”This is what they want!”

And now we begin to look for strategic and tactical recommendations. Now we can help close the gap between what the consumer wants and what the client makes.

The dangerous power held by the interviewer

A recent episode of This American Life tells a fascinating and horrifying story of a murder confession gone wrong. The story is a reflection from the retired detective who seems to have sincerely believed the woman in question to be guilty. He realizes in his reflection that he was open to hearing what fit his theory and dismissed information or cues that didn’t support his theory (this is known as confirmation bias). This is a real concern for people doing user research who have preconceived notions about people, their behavior, their desired solutions, etc. One tactic is to develop greater self-awareness and learn to hear your own biases and assumptions.

Even more disturbing in this story is how the suspect began to provide details of the crime that supposedly only the person who committed the crime would know. In fact, this woman who would want to clear her name, responded to the questioning by shifting to please her interrogator, looking to provide the “right” answers. While the police didn’t realize it, she was picking up clues from the documents they were showing her and presenting them back as if it was her own knowledge. She wasn’t trying to confess, she was trying to succeed in answering the questions, even though it was significantly against her own interests. This is also a crucial concern for user researchers, where participants will want to please them and will work hard to figure out what “pleasing” looks like. The way you ask questions (e.g,. “Do you like doing it this way or would you rather have it happen automatically when you enter the store?”) has a tremendous influence in how they are answered.

The New York Times offers this summary

He tells about a woman who confessed to killing a man. She knew insider things like that the victim was wearing his wedding ring when he died, and that his credit card had been used at a People’s drugstore and a Chinese takeout place. Case closed.

A few weeks go by, and it turns out the woman has a strong alibi. Charges are dropped.

Years later, with the case still officially open, Detective Trainum went back to the file because he still suspected that the woman had gotten away with murder. He discovered that he and the other detectives accidentally videotaped the whole interrogation — not just the confession. That’s when he found out how an innocent person could know unreleased details of the killing.

At one point during the interrogation, they were trying to get her to admit to using the dead guy’s credit cards, and said, isn’t that your signature on these slips? And they showed them to her. So she read the name of the drugstore and the restaurant.

At another moment, they showed her the crime scene photos. In one, the left hand of the corpse was prominent. You could see the wedding ring.

So they had accidentally fed her all the incriminating details that she returned to them in the confession.

Stories fuel listening

StoryCorps vehicle
StoryCorps is “an independent nonprofit whose mission is to provide people of all backgrounds and beliefs with the opportunity to record, share, and preserve the stories of our lives.” And they’ve just turned 10. Founder Dave Isay reflects

It was about a year into this thing when I began to, I think, fully understand the power of this very, very simple idea and decided to devote the rest of my life to building it into something that I hope eventually in this country moves the needle on getting people to listen to one another.

Yes, that’s right, an organization that is nominally about getting people to tell stories is really seeking to improve listening. Storytelling has that power; it leads to listening. The fuel for listening isn’t silence, it’s stories.

Steve interviewed for UX Magazine

logo

When I spoke recently in Los Angeles I was interviewed by Luke Swenson of Media Contour.

The interview has been published on UX Magazine.

When should our clients invest in user research?

There are two things that you should watch for that can indicate it’s time. One is when you realize that you don’t know the answer to an important question. For instance, maybe you’re not sure who you’re targeting with a product or service. The second one is more difficult: it’s when you believe you have the problem already solved but you’re operating without any type of humility—you’re believing your own hype, so to speak. You’re making assumptions without any facts or evidence to back up those assumptions.

What Data Can’t Do

heads
Heads, Oakland’s First Friday, June 2013

I do love this NYT Op-Ed (What Data Can’t Do) from David Brooks, especially

Data struggles with context. Human decisions are not discrete events. They are embedded in sequences and contexts. The human brain has evolved to account for this reality. People are really good at telling stories that weave together multiple causes and multiple contexts. Data analysis is pretty bad at narrative and emergent thinking, and it cannot match the explanatory suppleness of even a mediocre novel.

Of course, the qualitative narrative data is still data. It’s mostly the kind of information I work with. Perhaps it’s easy to conflate data with Big Data and succumb to the notion that if the approach to Big Data is limiting, then the approach to data in general must be limiting. But data is data – it’s what we use, in whatever forms, to inform and inspire and drive decisions.

The ethnographic research community is looking at the increasing reliance on quantitative data in business and questioning their role. Rich Radka proposes a “Yes, and…” mindset in this posting, no doubt one of many we’ll be seeing, as our business culture (and culture overall) evolves.

From my Los Angeles presentation on Interviewing Users

I had a wonderful trip to Los Angeles last week so speak at a combined IXDA Los Angeles/LA UX Meetup event. They gave me a really warm welcome (including a pint of cold Ben and Jerry’s ice cream all to myself) and the at-capacity room was filled with enthusiastic and thoughtful folks who contributed to an interesting discussion.

Here are the slides

And the video

Also, an alternate video is here and highlighted tweets are here.

Steve interviewed for UIE Book Corner

interviewing-users

Interviewing Users is now available. Get your copy here!

I spoke about Interviewing Users with Adam Churchill for UIE Book Corner. Listen at the link, or below.

The transcript is here.

Adam: What do you think people are going to do differently after reading this book?

Steve: I think the book sets people up-it sounds maybe a little trite, but let me try to unpack it a little bit-it sets people up to really hear the people they talk to, their users, their customers, their target, to really hear where they’re at and to understand what’s important to them, as opposed to sort of hearing what we want to hear.

And so this is where I said earlier there’s some philosophy, and there’s some tactics. And the tactics come from the philosophy. I think this is what I’ve seen lacking in practitioners that I meet with over the years, is that it’s sort of easy to ask questions or even ask questions well and hear what people are telling you.

But the broader approach, which is to understand people’s world as it’s organized and kind of structured and labeled from their point of view-it never matches either the frameworks that we have going in or the architectures of tools that we’re providing.

So to take that kind of deeply, completely user-centered approach and understand what the user’s building up their world out of, how they think about it and feel about it and talk about it, I mean, that’s what starts to be where you get towards excellence types of user research.

So that’s not what everyone’s always going to do. That’s not always what the objective is, but I think-I’m hoping that this will start to open up people to be able to do that, to have that kind of be baked into their approach to users. It’s not just collecting responses to questions but really grokking where they are coming from and how they’re operating.

Interviewing Users reviewed in QRCA Views Magazine

interviewing-users

Interviewing Users is now available. Get your copy here!

George Sloan has reviewed Interviewing Users in QRCA (Qualitative Research Consultants Association) Views Magazine. You can see the entire issue of the magazine here or read the review as a PDF here.

From an experienced researcher’s point of view, Interviewing Users is confirmation of the skills you have developed over the years, written from a research professional’s point of view. From a corporate researcher’s point of view, this book is strong armament to bring the design team into the research process with actual users, rather than thinking that they (as product users themselves) know all the issues involved.

Steve interviewed in UX Matters

uxmatters

UXmatters has just published their interview with me. Here’s a snippet; please see the whole thing here.

Jeff: Someone reading the unassuming title Interviewing Users might at first think that you’ve written an impersonal reference book. While there are sample documents and practical tips aplenty, there are also what are almost philosophical passages that refer to the self-control and mindfulness that a successful interviewer must cultivate. For example, you explain how to accept awkward situations and “check your world view at the door.” Do you feel that the many years you’ve spent practicing these skills professionally have bled into your interactions away from work?

Steve: Your question makes me happy. I think that was something I could bring to the book that might take it beyond a catalog of tips and tricks. The work that I’ve done and the opportunity that I’ve had to reflect on it over many years has given me a richer perspective. And you’re right, it comes down to a lot of fundamentals about who we are and how we deal with other people. I don’t mean that one can literally replicate the interviewer persona in every context-and to an introvert like I am, that would be a horrible idea-but what you’re saying is right on. My work has given me a lot of tools with which I can look at myself in other kinds of settings-both in social settings and in other kinds of professional settings. That’s obviously not the thrust of the book, but it’s there for the taking if you want to reflect a little bit on who you are, how you interact, and how that can inform your design work, your creative work, and so on.

Interviewing the Interviewer – Steve Portigal talks with Maish Nichani

index

UX Booth has published a two-part conversation I had with Maish Nichani of Pebble Road.

In Part 1, Maish interviews me (excerpt below). We explore a few aspects of the research process, including how a project plan is negotiated.

MN: What is an appropriate response to give clients who insist on specifying aspects of your research methodology?
SP: Whenever a client approaches me and has already specified the approach we should take with their study, that’s usually time for a conversation. Sometimes teams create a research plan as a stake in the ground when what they actually want is feedback and a recommended approach. Sometimes, though, their plan is a good one, and we might just suggest one or two changes to see if they are amenable. I can’t count the number of times I’ve received a detailed request, exclaimed “what?!” and then had a really excellent conversation to better understand the reasons behind it. Obviously, no one should take a project where they don’t believe the method is going to produce results. An examination of a prescribed approach is one of the first tests of (the potential for) good collaboration.

In Part 2, I interview Maish (excerpt below). We talk about how to improve the organizational learning user research.

SP: If you could wave a magic wand and create any kind of tool or artifact to support the research process, what would it be?
MN: Research is really all about creating new knowledge and the more people who have access to that knowledge, the better. Currently, our research findings and insights are all locked up in (what Karen McGrane calls) “blobs.” We need it, instead, to be “chunked” (as Sara Wachter-Boettcher says) so that it can travel more freely and be mixed and mashed up to create, again, new knowledge. I don’t know of any existing project or initiative but I was thinking about using a schema (like what is already available on schema.org) for research findings. That way anyone writing up research findings could use the same markup and then search engines and specialist apps could read and move those findings more efficiently.

User research self-reporting in the YouTube era

Pizza in JAPAN (embedded below) is a charming video by some Canadian students living in Japan. It’s the type of self-reporting I would love to see more of when I do researcher with users. In the easygoing video they walk through the whole experience from ordering online to the tasting. The young women offer pretty basic but helpful compare-and-contrast commentary. There are tons of YouTube videos showing just how crazee things are in Asia; this isn’t trying for that, it’s just such a nice explanation from someone in a new culture about that culture, speaking back to their home culture.

I’m curious if an increasing fluency with digital tools mean that we can start to expect this level of quality in self-reporting from engaged, creative research participants. Self-reporting is of course limited to what respondents decide to share with you; it’s always going to be incomplete but I see this wonderful example as very encouraging. Check it out: look for the little Japanese mannerisms and marvel at the experience these girls are having.

This is What PIZZA in JAPAN is Like 🍕

The art of the interview

Here are two insightful takes on the art of interviewing, from two different sources.

First, Ira Glass is interviewed by Jacob Weisberg (the short video is embedded below). Glass explains how he helps people feel comfortable sharing with him by bringing himself into the conversation (a technique I’m not so keen on for user research, although I’ve seen some people be successful with it). He also reveals that what is edited out of the broadcast interviews are tons of clarification questions, where he’s following up to understand the sequence of events, or the different people involved in the story, etc.

Second, How to Listen makes a good case for the authentic personal elements that we ourselves bring to our interactions with interviewees.

Dr. Mason had a simple method of getting me to begin. He would lean slightly forward, all the while maintaining eye contact and then when he got my attention, he would nod. I will never forget that nod; it was a signal that he was with me and I could safely express myself about whatever was on my mind, but I realize now that he was controlling the conversation. A cursory nod encouraged. Elongated ups and downs, (and the raising of eyebrows!) symbolized agreement.

This is the first lesson for writers – or anyone – who conducts interviews: If you want someone to talk, you’ve got to know how to listen. And good listening is a surprisingly active process. The interviewee is your focus of attention; you are there to hear what he says and thinks, exclusively. When I say, “interviewing,” I am talking from the perspective of a narrative or creative nonfiction writer. Interviewing for news is somewhat different; reporters usually know, more or less, the information they need to unearth. The writer of narrative, by contrast, is often seeking the unknown – the story behind the facts. You won’t always know the story until you hear it; your job as an interviewer, often, is to keep your subject talking.

On Huffington Post – An Interview with Steve Portigal

Check out An Interview with Steve Portigal

Q. What’s the difference between this type of research and something like focus groups?

A: When interviewing people we arrange to meet with them in their own context: at home, at work, at the park, in the car; wherever the thing we’re interested in is happening. We’ll be with them and maybe some of the other people you would find in those settings. People behave differently in their own environment and there are details of those environments that turn out to be relevant but we could never plan for. The organized pantry reveals something about how they approach the apps on their device, the room full of the father’s previous generations of PCs tells us about the son’s rejection not only of specific devices but also a whole approach to what it means to own devices.

Q: How important is empathy for organizations?

Empathy seems to be a hot term right now. Developing processes that include empathy is wonderful and it can really help teams rally around solving the problems that people have rather than the problems they want to work on. You hope that there’s an overlap but not always. Maybe this points to what school of innovation you’re from; you may want to be a few steps ahead of the people who will be your customers but you do need to be thinking about where they will be headed.

Empathy, meanwhile, is only the first step. Having a good sense of how people feel can instill the desire to do the right thing for them, but it doesn’t tell you what that right thing is. Empathy is not the same as understanding a highly nuanced, unarticulated, latent problem space. That’s the hard work.

I, For One, Welcome Our New Robot Interviewer Overlords

From Wired, comes These Adorable Robots Are Making a Documentary About Humans. Really.

Created by artist and roboticist Alex Reben for his master’s thesis at MIT, the BlabDroids are tiny, adorable robotic cinematographers who will be filming interviews at this week’s Tribeca Film Festival in New York as part of the the film festival’s transmedia Storyscapes program. At least 20 BlabDroids will zip around to attendees-they’re self-propelled via motorized wheels- and ask them often very personal questions like, “Tell me something that you’ve never told a stranger before,” “What’s the worst thing you’ve done to someone,” and “Who do you love most in the world?”

Each droid carries a digital camera, a speaker that asks a series of pre-programmed questions to ask whomever it encounters and a button to be pushed to prompt new queries.

“We plan to give the robots to some interesting New Yorkers,” filmmaker Brent Hoff, who is working on the BlabDroid project with Reben, said in an email to Wired. “Hopefully Anthony Weiner and some Broadway types.” The robots, which are very adorable and voiced by a 7-year-old boy, are intended to test the theory of MIT computer scientist Joseph Weizenbaum’s “ELIZA effect,” which found that people are inclined to anthropomorphize computers and thus engage emotionally with artificial intelligence. Although this initially lead Weizenbaum to worry about AI’s potential to manipulate, Reben and Hoff have created the BlabDroids to appear comforting and non-judgmental, and to capture meaningful interactions with their subjects.

This is a fascinating experiment. I’d love to see the results, as well as the raw footage and of course to get to talk to someone about their experience being interviewed like this would be very interesting. Similar to Interviewing without questions, eye contact or rapport, the notion of freeing up people from human interactions in order to liberate them to reveal more deeply is a curious (and certainly valid) idea. No doubt the clips will be great; people are pretty interesting and curious, so pointing a camera at them will be entertaining.

Ethnography of Elevator UX

Check out An uplifting experience – adopting ethnography to study elevator user experience (on Ethnography Matters). I love that someone has done (and shared) a study like this.

A few years ago a leading elevator design and manufacturing company gave me the task of examining how people experienced and interacted with elevators. The scope included everything from hall call buttons, to cabin interior design and perception of technical design. When given the brief, the artistic director noted country specific design features (or omissions) and even mentioned that there may be observable elevator habits I would want to take note of. Then, on our bidding a corporate-academic farewell she added that I might want to consider the psychology of the surrounding architectural environment. With that, I was left with a long list of to-do’s and only one method I could think of that would be capable of incorporating so many factors – ethnography.

I’m reminded of Elevator Pitch (PDF) our 2011 article in interactions that considered different types of elevator user experiences and mused about the implications for approaching design problems.

Rethinking Everything About What You Do For Customers

kiss

Rebecca Mead’s New Yorker feature The Sense of an Ending describes some really dramatic (and successful) reframes in care for dementia patients. There’s a number of profound shifts in how the caregivers describe their role and in the kind of experience they seek to deliver for the patient (and their family). The whole article (linked above, but subscribers only) describes those shifts and the cultural and organizational efforts to get there. I’ve included just a portion here

One of the first things Alonzo did, in 1998, was to ask an aide who was born in Vietnam to talk to staff members in her native tongue. “It was the only language I could find that nobody else could speak,” Alonzo recalled. “So we had her tell us very sweetly, in Vietnamese, what she wanted us to do, and we couldn’t understand her.” The staff had to become attuned to the woman’s nonverbal cues.

On another occasion, Alzono underwent a public bed bath, in front of the entire staff, of twenty-seven. She didn’t allow herself to move her limbs, and behaved as if confused. Afterward, she was able to describe the nature of her discomfort, and staff members analyzed their own activity in light of it. “Let me tell you, it sucked – it was incredibly uncomfortable,” she told me. Staff members then spooned food into one another’s mouths and brushed one another’s teeth, in order to be on the receiving end of activities that they performed for their charges every day. “You can find how threatening it is to have something touch your mouth when you have not brought it to your own lips,” she said.

In the most radical experiment, the staff wore adult diapers. “That was kind of life-changing for everybody involved,” Alonzo told me. “We all recognized just how uncomfortable it was to sit in a wet brief. Some of our front-line staff, who really wanted to know how bad that felt, did not change them for a couple of hours.” Previous may residents had been dressed in diapers, as they tend to be in a majority of nursing homes. Not long afterward, aides decided to stop the practice with most residents, instead taking them to the bathroom fifteen or twenty minutes after mealtimes. This made residents happier while making the staff’s jobs easier, because they no longer had to change people who were agitated.

There’s a rich tradition of participating in the experience our customers are having (see this great war story about an adventure in an “old age simulation suit”) and what feels like an increasing mention of empathy. I really like how this story highlights not so much the ergonomic or functional task aspects that are revealed but how this drives to revisiting the fundamental ideas of how the institution conceives of the patient experience it provides. I also like the full-on simplicity of the approach, the people who do this stuff to others now try it themselves and talk about it.

See also Richard Anderson’s blog post from this week about reframes in general and in healthcare specifically.

Pattern-recognition is crucial for sense-making

interviewing-users

Interviewing Users is now available. Get your copy here!




theartofscientificinvestigation


Excerpting from a great post…The Art of Observation and How to Master the Crucial Difference Between Observation and Intuition [Brain Pickings] – Highlights from a 1957 book by Cambridge University professor W. I. B. Beveridge come from the era of the scientific method but are broadly applicable to creative, innovative, design-thinking approaches to problem solving.

Ultimately, Beveridge argues that the art of observation depends on developing the capacity for pattern-recognition, which in turn relies on a broad pool of networked knowledge that allows you to spot the piece that doesn’t fit: “In carrying out any observation you look deliberately for each characteristic you know may be there, for any unusual feature, and especially for any suggestive associations or relationships among the things you see, or between them and what you know. – Most of the relationships observed are due to chance and have no significance, but occasionally one will lead to a fruitful idea.”

Harry Dean Stanton and Silence

interviewing-users

Interviewing Users is now available. Get your copy here!



hds
At SXSW this year we saw Harry Dean Stanton: Partly Fiction, a moody and elliptical portrait of the actor. Despite the filmmaker being a long-time friend of his, Stanton is evasive and mercurial, seeming more earnest when performing music for the camera than in answering questions about his parents or his relationships with women. On more than one occasion, the subject doesn’t respond and just stares off or at the camera or the interviewer. And the interviewer stays quiet for a surprisingly long time.

There was a Q&A after the screening, so I asked the filmmaker about what she thought about the power of silence (for in addition using silence in the interview, they also chose to leave those silences in the final film). She told us “I just wanted to see what would happen, and to see the boundaries of being uncomfortable.”

I found this fascinating; in Interviewing Users I describe a scene from Werner Herzog’s Grizzly Man where the director uses silence to gives space for an astonishing and revelatory moment to occur. In this case, “nothing” happened. Of course, as the director reminded me, the “nothing” that happened with Harry Dean Stanton was still something; it revealed a lot about the subject and changed our own experience in hearing his story.

It’s further illustration of the power of silence, even when it doesn’t pay off in the obvious manner and bring something out, it’s still bringing something else out!

Empathy begets empathy

interviewing-users

Interviewing Users is now available. Get your copy here!



In Chapter 1, I argue that in addition to gaining valuable information through interviewing, the process also builds empathy which in turn increases our overall capacity for empathy.

Marc Maron (describing his journey with the WTF podcast), in this Rolling Stone interview, says the same thing in his own way.

But once I started talking to people, I evolved a capacity I never had before, which was to be an empathetic listener. I still step on people a lot, and I interrupt them with my own bullshit. But I was a better person. I was humbled.

Also0 see yesterday’s Maron example here.

Marc Maron on listening

interviewing-users

Interviewing Users is now available. Get your copy here!

About a year ago, Marc Maron (comedian, host of the WTF podcast, and an example in Interviewing Users) was on Chris Hardwick’s Nerdist Podcast. They got into a discussion of interviewing, touching on listening-builds-empathy, the pressure on the interviewer to keep talking and choosing to interrupt.

MM: It’s one-on-one, we’re in a garage, it’s cozy, and I’m pretty much a too-much-information kinda guy and I’ve learned how to listen. I don’t know when that happened. I’m happy it happened but -when I was a kid I grew up with a bipolar father so there’s some part of me that’s wired to emotionally connect with very charismatic volatile people and throughout most of my childhood I would always be attracted to crazy homeless people that walked around where I used to work at this restaurant. I always wanted to hear their stories. I always liked people that were funny and large personalities because I’d been sort of diminished by my father’s brutality in terms of his emotions. And I just find for a long time there I was bitter, and I was “Aw fuck, everyone was out to get to me” and defensive and I’d gotten very cynical and now because of the podcast I’m very empathetic. When I listen, there are times when I’m about to cry, or where I’m so engaged in the story that I’m just happy when -wow I’m into this.

I think that is just my own curiosity, the way I’m wired but I’m very happy that’s happened. I like to listen to people.

CH: It’s something that I certainly I wish I did more sometimes, I’m always like-NO DEAD SPACE, NO DEAD SPACE – always gotta keep talking

MM: I think that’s natural, I do that..Some people criticize for me on my podcast that I occasionally interrupt too much, it’s because they’re not used to listening to interviews the ways I do. A lot of times I interrupt because I get nervous if people are doing some – if someone is a public personality, they’ve got their story, you know, and I can sorta tell when like I’m not the first one to have heard this story, sometimes – it’s happened with Anthrony Bourdain specifically because I wanted to connect with him, but he speaks in public a lot and I listened to part of his interview on Joe Rogan, one because I wanted to see what kind of person he was in conversation and I knew there was a couple of points that he clearly hit regularly so when I heard him start those I was “Yeah, but what about-” I tried to get around them only because I wanted to see if I could get-I ended up getting something different, it was good, it was good. But I’m sorta aware of that stuff. You?

CH: I try to be aware of it, but I also try to keep the conversation going and a lot of times if I’m talking with someone and this is why I don’t do phone interviews because I need to see people, I can tell they are close to finishing a thought and I just don’t want any of those moments where like-if you listen to this podcast you will never hear a [pause} “So-.”

Steve interviewed by Denise Lee Yohn about Interviewing Users

I was interviewed by Denise Lee Yohn – we chatted about interviewing, insights, innovation, iPods and probably some other words that start with the letter i.

Listen at the link or through the widget below.

To download the audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac).

Body language changes you inside

interviewing-users

Interviewing Users is now available. Get your copy here!

In Chapter 2, I talk about how body language (see good and bad examples here and here) not only signals that you are listening to your interviewee, it also signals you to listen better. I based on this on some writing by Malcolm Gladwell (in Blink, but originally in The Naked Face) about how our physical self can induce changes in our emotional selves.

Now there’s more research to back up that claim. From this WSJ article

Researchers are finding that wearing a smile brings certain benefits, like slowing down the heart and reducing stress. This may even happen when people aren’t aware they are forming a smile. The work follows research that established that the act of smiling can make you feel happier. Frowning also may have a health effect: Preventing people from frowning, such as with the use of Botox, can help alleviate depression.

“You can influence mental health by what you do with your face, whether you smile more or frown less,” says Eric Finzi, a dermatologic surgeon and co-author of the study on frowning.

Bonus: the article includes this compelling image, explaining “Holding the sticks in the mouth activates the same muscles we use for smiling.”
chopstick-smile

Interview with Steve posted on Ethnography Matters

Here’s an interview with me at Ethnography Matters. We talked about the book, the writing process and other aspects of how interviewing users is playing out in the corporate world.

EM: How much heterogeneity is there between companies / clients? Are there any broad typological characterizations of companies and their attitude towards user research? Does this inform different ways of delivering research results, different ways of “talking to” these companies?

SP: There are no doubt dozens of frameworks (see for example, Jess McMullin’s Design Maturity Model on page 142) for characterizing the organization. But let me throw out a new one: in the Passover Haggadah there is the example of the four sons. One wise, one wicked, one simple and one who does not know how to ask a question. The wise son asks to have all of the history, insight and other findings explained to him. We’re encouraged to explain everything to him. The wicked son separates himself from the issue by asking why it’s important to you. We’re told to tell him why it’s important to us (and not persuade him that it should be important to him). The simple son doesn’t even focus in on the issue and just asks “What is this?” so we’re to give him the headline. We’re told to approach the son who doesn’t even know how to ask a question and take the initiative to explain things to him. And one scholar writes about a fifth son who isn’t even in the room and it’s up to us to seek him out and give him the lowdown.

Sure, it may be a bit forced but it’s not hard to see those sons as archetypes of individuals, departments or entire workplace cultures. Whatever your framework is, you obviously need to understand the specifics of who you are dealing with and have a range of approaches for responding. All of this stuff with people (be they clients or research subjects) is messy and I’m not so comfortable with pre-emptive categorization and its resultant tactical choices.

Thanks to Tricia Wang and Jenna Burrell and everyone else at Ethnography Matters for a great discussion.

Interviewing without questions, eye contact or rapport

interviewing-users

Interviewing Users is now available. Get your copy here!

Here’s a really interesting project about How People Talk to Themselves in Their Heads.

He would ask them to wear a microphone headset attached to a digital recorder and speak aloud their thoughts as he followed closely behind with a camera. He would not be able to hear what they were saying…[The] videos are simultaneously naturalistic and as objective as possible. In the lab, in front of a researcher, people are often reluctant to reveal exactly what they are thinking. Writing a diary of inner speech is somewhat more private, but many people find it annoying to regularly drop everything and make an entry; sometimes it’s difficult to remember what one was thinking about even minutes earlier. In Irving’s videos people are living their lives more or less as usual, walking and talking to themselves as though they were unaccompanied. Of course, people who are not completely comfortable with the scenario sometimes speak into the microphone as though trying to entertain someone else. And getting people’s inner speech on tape captures only linguistic forms of thought, neglecting the kind of thinking that happens in images and scenes, for example.

The notion that unfettered, deeper self-exploration and self-expression can come when not interacting with the interrogator – to the point here of nearly eliminating the interrogation entirely – evokes the (I presume mostly obsolete) approach to therapy where the patient does not face the therapist.

I find the videos compelling (and voyeuristic to the extreme). Check out the other videos at the link – the ones that take place without the strolling seem more like a diary and less like a peek into the stream of consciousness. But for each of them, see if they pass the sniff-test for you: is the person talking the way they are because of the experiment (they know they are being recorded; they feel they need to come up with something to say, they are aware of their own “voice”, etc.) or is it really getting as deep as the researcher claims? I was mostly convinced but a sliver of doubt remains.

In the research we do, we make no claim of naturalism; we certainly want to direct and influence what is being shared, and we build rapport to facilitate openness and honesty. This approach isn’t likely to be appropriate for us, but it’s certainly provocative to look at the output of an opposite approach – where the interviewer is effectively absented and rapport is not a consideration.

Loss of Context III

From New York Times Corrections for April 7, 2013 comes another demonstration of a) the importance of context in understanding what interview respondents are saying and b) the necessity of an actual recording of what was said.

An accompanying feature transcribed incorrectly a comment from Callie Khouri, creator of the television drama “Nashville,” about what she would put on her Easter playlist. Khouri said she would include music by Pops Staples, the late patriarch of the singing family the Staple Singers. She did not say she would include “pop staples.”

I share a story of how this confusion can easily happen in fieldwork in Conversational Layers and have some other examples from the media in Loss of Context I and Loss of Context II.

Steve interviewed by Tomer Sharon

steve-and-tomer

I was interviewed by Tomer Sharon (as part of his incredible series of interviews for It’s Our Research). Our conversation ranged from about why it’s hard for people to do user research, the collaboration between agency and clients, and how to think about organizational and stakeholder challenges as design problems.

The 22-minute video is embedded below.

Steve Portigal It's Our Research interview

Video from UX Lisbon: Discover and act on insights about people

The lovely folks at UXLx have just posted the video from my talk earlier this year, Discover and act on insights about people.

Some of the most effective ways of understanding what customers want or need – going out and talking to them – are surprisingly indirect. Insights produced by these methods impact two facets of innovation: first as information that informs the development of new products and services, and second as catalysts for internal change. Steve discusses methods for exploring both solutions and needs and explores how an understanding of culture (yours and your customers) can drive design and innovation.

If you don’t see the video embedded above, you can view it here

People Like Us

People Like Us was a mockumentary that ran on BBC radio in the late 90s before becoming a TV show. The radio shows are hilarious and a great illustration of what can and does go wrong in fieldwork. Each episode is essentially a total War Story.

The shows follow a hapless reporter, Roy Mallard, investigating the lives and work of ordinary people: a bank manager, an artist, a stay-at-home mom, an actor and so on. Things go awry: despite being married (Really? You?) he finds himself awkwardly attracted to an interviewee, only to realize that another interviewee is her bitter ex-boyfriend. He’s a passenger in a recklessly driven car. He’s witness to firings, incompetency, violence, relationship hassles. He trips, drops things, is sneezed upon, breaks a washing machine, and more.

At the same time, his attempts to interview people and get to the heart of what their lives are about are thwarted. If not by circumstances, then by the inarticulateness of his interviewees, or by their sheer misinterpretation of his questions (e.g., Q: You’ve been here for a long time. What kind of things have changed? A: My hair.)

People Like Us manages to be completely absurd yet with an eye-rolling kind of truth that any user researcher (and journalist, I imagine) will identify with.

The radio series has been posted to YouTube and I’m embedding all the episodes as a playlist below. Check ’em out and let me know what you think.

Interruption or Interjection?

Deborah Tannen writes in the New York Times about interruptions. She’s riffing on this week’s presidential debate but I thought this part was relevant to interviewing:

You might think it’s obvious that an interruption is when a second person starts talking before another has stopped. But how long a pause means “I’m done” rather than “I’m catching my breath”? This, too, varies by region and culture – and the difference can lead to unintended interruptions. In 1978, I tape-recorded a Thanksgiving dinner conversation involving two Christians raised in California, three Jews of Eastern European ancestry from New York and a British woman. At times the Californians felt interrupted when their Jewish friends mistook a pause for breath as a turn-relinquishing one. At other times, exclamations like “Wow!” or “That’s impossible!” which were intended to encourage the conversation, stopped it instead. An interruption takes two – one to start, the other to stop. The New Yorkers in my study assumed that a speaker who wasn’t finished wouldn’t stop just because someone else started. If she does, then she creates the interruption.

In my book I look at interruptions and turn-taking in interviews. If someone is going on and on and we need to redirect them, how to do so elegantly? If we are having trouble not talking over someone, what are the sources of those missed pauses and cues? Tannen’s exploration of interruption is useful fodder for thinking about this.

Henry Thomas and the Power of Silence

Henry Thomas’s audition for E.T. is making the rounds in the blogosphere this week. It’s a pretty incredible bit of emoting, improvising, acting. I’d like to highlight one thing that struck me: as the scene begins, the offstage actor sets up the conflict with his first line, and then turns it over to Henry (aka Elliot). And Henry doesn’t say a word. But he does a lot. His silence is very active, even his minimal facial movement is highly active.

In interviewing users, those small moments where we choose to be quiet are powerful. We can maintain an active engagement with the participants even though we aren’t talking. We can move the interview forward without using our mouths.

Rapport building

Friend-of-a-friend Elizabeth Rubenstein took this awesome picture at our local Safeway.

I’m always amazed to see the backstage on display where those of us who are frontstage can see it (see another Safeway example here). In this example we’ve got two separate Rapport Topics Of The Day:

  • How do you like todays (sic) weather?
  • How do you think the Giants baseball team is doing?

Safeway has a long history of awkwardly conceived inauthentic rapport-building techniques, such as the one I wrote about back in 2002 where staffers would hold onto my receipt for a painfully long time while they tried to puzzle out the pronunciation of my name, before handing it back after muttering “Thank you, Mr. Portugal.”

For what it’s worth, they seem to have got better with the name thing, and I haven’t been asked any false-note questions about the weather or the Local Sports Team.

Other Safeway goodies from the past:

(Thanks to Jen Lum)

Dark Patterns for Interviewing

How to Win at Conversation is a humorous New Yorker article that frames conversation as a competition and offers up strategies (including Seed of Doubt, Barrage of Interruptions, Intentional Mishearing and Unfulfilled Intimations of Actual Gossip) for winning. You’ll likely recognize when you’ve been on the receiving end, or maybe when you’ve done it yourself. The piece can be read as a set of dark patterns for good interviewing.

OPPONENT: We just got some pretty good news.
YOU: I can’t believe it! They finally gave you a five-hundred-thousand-dollar raise, didn’t they?
OPPONENT: Er…no.
YOU: Oh. Sorry. So what’s the good news?
OPPONENT: Our little Jimmy just got into his first-choice preschool.
YOU: Oh. That’s good, too. Certainly nothing to sneeze at!

Strategy used: Intentional Overstatement of Expectations.

I’ll tell you something I think you’ll understand

Losing All Hope Was Freedom (or LAHWF) is a project (with an associated YouTube channel) devoted to breaching experiments (“an experiment that seeks to examine people’s reactions to violations of commonly accepted social rules or norm”). It takes a certain bravery to engage in these experiments, say going into an elevator and facing the back wall, to see what others will do. It also hopefully requires some ethical examination; some of these experiments come off more as pranks and can disturb people (honestly, I’m not impressed with Improv Everywhere because they needlessly perturb strangers for our bemusement).

With that in mind, here are their two latest videos, where they sneak up on people and casually take their hands. Compare the experimenter and the environment between the first and second video, and look at your own reactions to the reactions they gather. The second one, for sure, is worth watching all the way through.

Holding People's Hand

Holding People's Hand 2

Observation and empathy

Here’s another proof point for the power of video in user research. Check out this very simple observational video.

If you didn’t watch it, it shows person after person stumbling on poorly designed stairs.

I don’t know about you but I felt increasingly emotional the more I watched this. A bubbling outrage and a sense that something so obviously needs to be done about this. Of course, this is a simple problem, which makes the failure to act even more aggravating.

The goal of user research isn’t always to uncover people’s fail states with the team’s existing products, but when it is, tools like video are impactful on rational and emotional levels.

Update: according to this Tweet, the stairway is now closed.

Our latest article: Never Eat Anything Raw


Our latest interactions column (written by Steve Portigal and Julie Norvaisas) Never Eat Anything Raw: Fieldwork Lessons from the Pros has just been published.

Interviewing is based on asking questions. As children we all learned to ask questions (perhaps more than the adults around us were ready for!), but it takes work to become a skilled interviewer-the kind of interviewer with whom a natural exchange is almost inevitable and for whom asking questions is as effortless as Roger Federer’s forehand. Great interviewers are made, not born. We’ve had the ongoing opportunity to think even more about the experiences that have shaped us as interviewers. Of course, as researchers we are compelled to look outside ourselves, so we asked some people we admire to tell us about how they improve their interviewing skills. We have synthesized our findings into four key areas: practice, reflect,
critique, and exchange.

Get the PDF here, and check out (and add to) the fieldwork War Stories here.

Previous articles also available:

Playing Participant: An autoethnography

With our Curating Consumption series Steve and I take time to look through our researcher lens at our lives as consumers. Sometimes we get to play participant and experience the other side of the research conversation. I recently participated in an online focus group for the redesign of a website. I couldn’t help but reflect on my own experience in contrast to that of being on the other side of the virtual glass. While the opportunities below were generated from a moderated online context, they also suggest possibilities for designing real time research interactions:

What Was Happening: I logged in and waited for the moderator to start talking. This was a silent discussion. Everyone was typing.

What I Was Thinking: Here I am at my computer, wearing my ear buds, ready to listen and there’s no talking. Oh my God. I’m an extrovert. How can I make it through 90 minutes of silence? How can I get my big wordy thoughts into these little text boxes? This is not what I was expecting.

Insight: No one set appropriate expectations for what ‘online focus group’ meant. I assumed it would be like a focus group with actual verbal communication. As an extrovert I found it difficult to sit quietly for 90 minutes with a virtual room full of people. As a verbal processor I struggled to articulate some of my ideas as typed words with limited character restrictions. This may have felt considerably different had I known going into it what to expect.

Opportunity: Use words wisely. If you call something X but it is different from most Xs, then clearly communicate how it is different so participants have appropriate expectations. Or don’t call it X.

Opportunity: Employ a variety of methods that cater to diverse personalities (i.e. introvert, extrovert) and learning preferences (i.e. visual, auditory, kinesthetic). Try to avoid using only one mode of interaction, it can feel alienating and disillusioning for participants.


What Was Happening: The moderators introduced themselves and set some guidelines for the session. Our first instruction was to introduce ourselves with a story about our name.

What I Was Thinking: Fun! Our icebreaker is simple! A story about my name: where it came from, what it means, whatever I want to share! Am I allowed to talk to the other participants? I want to comment on that story!

Insight: This was a fun and simple icebreaker with a low barrier to entry (everyone has a name!). It was also appropriate to the context because all I knew of the other participants in the group were their names. We began addressing comments to each other and then the moderator encouraged us to do more of that. We quickly established a rapport and connected to each other through these stories.

Opportunity: Facilitate rapport building between researcher and participants AND among participants. If you have expectations about who should (or shouldn’t) be talking to who, clarify that at the outset (or before beginning).


What Was Happening: We were asked questions by the moderator (her type was in bold and blue) and then all of our responses would come up in a feed below. It was a small window that I was unable to resize or navigate.

What I Was Thinking: This interface is driving me nuts. I am struggling to follow all of the comments. The moderator’s questions get lost upstream when everyone starts answering. When I try to go up to revisit the question, an answer comes in and I lose my place in the thread. I cant’ find the rating scale we are supposed to use. Is 1 high or low? ARGH!

Insight: The researcher often has a clear path through the conversation in mind. Participants don’t necessarily have this big picture view and can feel lost in the forest of questions and answers.

Opportunity: Ensure participants have various tools for keeping up with the flow of the conversation. This may be easier in live/in-person meetings, and especially valuable for virtual or asynchronous interactions.

Opportunity: Provide a map of the journey that enables participants to identify where they are if they feel lost. Let them peek behind the curtain to see what’s ahead. It can be a trust builder if done well, or a spoiler alert if not framed appropriately.

If you want to play with the possibilities of using these opportunities to improve your own practice (I know I am!), you can turn each Opportunity into a question that catalyzes divergent thinking. Simply ask “How might we…” before each Opportunity (e.g., How Might We ensure participants have various tools for keeping up with the flow of the conversation?). Then challenge yourself to generate as many ideas as you can (20 is always a good round number). And if you do, please share! We would love to hear suggestions for how to improve the practice of research by improving the design of the participant experience.

Well, thank you for joining us

For a little Friday Fun, here’s Mike Birbiglia‘s new short film from This American Life LIVE (if you are in the US, Canada, or Australia, I highly encourage you to find a screening near you for this next Tuesday; truly a wonderful entertainment and storytelling experience).

In this short and gently comedic film, Birbiglia pokes fun at some the norms of interviewing (and being interviewed).

Fresh Air 2: 2 Fresh 2 Furious

Video now available from Steve’s talk at Mozilla

Last week I visited Mozilla’s beautiful, dog-friendly offices to talk with their user researchers and designers. They’ve just posted the video from my presentation of We’ve done all this research, now what?. Note that the start is cut off, and it kicks in at 11:47.

Note: the slides are included in the video but for easier viewing check out a similar presentation here.

Interviewing past the platitudes

From this past Sunday’s New York Times, a review of Craig Taylor’s “Londoners”:

Taylor devoted five years to collecting the material for “Londoners.” He gathered stories from all 32 boroughs, conducting formal interviews with more than 200 people, running through 300 tape-recorder batteries and taking down enough notes to generate transcripts of more than 950,000 words. Fewer than half the people he talked to made the final cut. Some interviews took months to set up and lasted just a few minutes. Others went on for hours.

The reviewer, a journalist herself, makes an excellent point about the challenges in interviewing:

Anyone who conducts interviews for a living knows how hard it can sometimes be to get subjects to move past cant and cliché, to leave the platitudes and drive on to the good stuff. (How many articles feature people expressing “shock and sadness” at their neighbors’ personal tragedies or noting that a murder victim “kept to himself”?)

Collecting soundbites is easy; the trope of the soundbite is so well-entrenched that most anyone will perform one in the right circumstances. Sometimes those soundbites feel – to the interviewer – like nuggets. It’s up to us to understand the difference between performance that we’re eliciting and a comment or observation that leads us towards insight.

Note: For more “Londoners” Taylor himself has an essay elsewhere in Sunday’s paper.

Seventeen types of interviewing questions

I’m cited in Developing Your Interviewing Skills, Part I: Preparing for an Interview, with a set of question types. The article suggests those question types are helpful in preparing an interview guide. I think they are also very helpful in the interview itself, as you will often have to probe a number of different ways to get at what you are think is interesting.

Anyway, I’m not sure where the author found that set of questions, but I’ve recently rewritten and restructured them for the book. This seemed like a great opportunity to share them with everyone. I’d love your feedback: What am I missing? Do you disagree? What else would be more helpful for readers?

Questions to gather context and collect details

  • Ask about sequence “Describe a typical workday. What do you do when you first sit down at your station?-Then what do you do next?”
  • Ask about quantity “How many files would you delete when that happens?”
  • Ask for specific examples “What is the last movie that you streamed?” – Compare this to “What movies do you stream?” The specific is easier to answer than the general and becomes a platform for follow up questions.
  • Ask for the complete list “What are all the different apps you have installed on your smartphone?” – This will require a series of follow up questions, e.g., “What else?” because few people will be able to generate an entire list of something with some prompting.
  • Ask about relationships “How do you work with new vendors?” – This general question is especially appropriate when you don’t even know enough to ask a specific question (e.g. in comparison to the earlier example about streaming movies). Better to start general than to be presumptive with a too-specific question.
  • Ask about organizational structure “Who does that department report to?”

Questions to probe on what’s unsaid

  • Ask for clarification “When you refer to “that” you are talking about the newest server, right?”
  • Ask about code words/native language “Why do you call it the ‘Batcave?'”
  • Ask about emotional cues “Why do you laugh when you mention ‘Best Buy?'”
  • Ask why “I’ve tried to get my boss to adopt this format, but she just won’t do it-” “Why do you think she hasn’t?”
  • Probe delicately “You mentioned a difficult situation that changed your usage. Can you tell us what that situation was?”
  • Probe without presuming “Some people have very negative feelings about Twitter, while others don’t. What is your take?” – Rather than the direct “What do you think about Twitter?” or “Do you like Twitter?” this question introduces options that aren’t tied to the interviewer or the interviewee.
  • Explain to an outsider “Let’s say that I’ve just arrived here from another decade, how would you explain to me the difference between smartphones and tablets?”
  • Teach another “If you had to ask your daughter to operate your system, how would you explain it to her?”


Questions that create contrasts in order uncover frameworks and mental models

  • Compare processes “What’s the difference between sending your response by fax, mail or email?”
  • Compare to others “Do the other coaches also do it that way?”
  • Compare across time “How have your family photo activities changed in the past five years? How do you think they will be different give years from now?” – The second question is not intended to capture an accurate prediction. Rather, the question serves to break free from what exists now and envision possibilities that may emerge down the road. Identify the appropriately large time horizon (a year? Five years? Ten years?) that will help people to think beyond incremental change.

Introverted Observers

We’ve had a lot of good posts – and comments – as of late about extroversion, introversion, talking to strangers, comfort zones, and so on. This brought to mind a story from a visit to New York a while back. In Let’s Embrace Open-Mindedness I tell two stories from my personal life (e.g., not when conducting research) where I explored the edges of my own comfort zone in just slightly unfamiliar circumstances, one situation where I saw the opportunity and couldn’t make the leap, another where I saw the opportunity and convinced myself to take that leap.

Followers of this blog will know I love taking pictures of curious and interesting things that I see everywhere, but it’s much harder – and not always appropriate – to take pictures of the curious and interesting people that I see everywhere. Indeed, in true Heisenberg fashion, you can’t always get the picture you’d want if you have to interact.

Anyway, visiting New York and walking through Times Square, I came upon people promoting Carrie Fisher’s “Wishful Drinking.” They were dressed as parodies of the book cover, with Fisher as Leia. At two separate points, I asked these hawkers if I could take their pictures.

Well, sure. It’s New York. It’s Times Square, thick with tourists, and these people are calling attention to themselves for promotion. All those cues shift the norm and make it reasonable/comfortable/appropriate/possible to do something that we don’t normally do: asking “Hey, can I take your picture?” That’s probably why I have so many photos taken with Shrek, Mr. Peanut, an Animaniac, the Monster.com monster – there’s something delightful and ironic about this staged naturalism, as if yes, I am hanging out here with my arm casually thrown around a 6-foot be-monocled legume. The opportunity to ask for a picture is so built-in to our scripts that it seems a crime to not get the picture!

Also see: The bear that saluted me

Talking to Strangers: Eugenio and Grace

Where I see boundaries, you see opportunities. – Steve, to me

On Monday Steve and I stumbled into a conversation that surfaced this difference between us in how we think about communicating with people. I’ve been reflecting on it all week and considering how it affects my interviewing practice. Mostly I have been paying more attention to how I am thinking during conversations and what kinds of opportunities I am seeing and looking for. Hot on the heels of Steve’s post with tips to improve interviewing skills, I hoped to surface a new point or two.

Yesterday morning I was walking a trail along the ocean. I heard a woman remark to the man next to her, “Well that was very creative of you!” I tried to keep walking, honestly I did. But I love creativity almost as much as I love talking to strangers so I had to backtrack- two loves in one conversation was irresistible! “Excuse me. Hi, I’m so sorry to interrupt you, but I overheard you say something about him being creative and I’m so curious! What creative thing was he doing?” So began my 20-minute interlude with Grace and her husband Eugenio (as Grace explained, “It’s pronounced ay-you-HEN-ee-oh”).

He is an artist, a painter. “I prefer mostly abstract and figurative painting. But you have to find your own voice-You can’t do too much school. I did some school when I was younger, in Mexico City. But if you are in school too long you become a mannerist. It just gets harder to find your own voice and be honest with it.” He told me about Joseph Beuys and Hockney (who Eugenio insists is overrated). We shared our mutual love of making art in and with nature. “You haven’t seen The Crack by Goldsworthy yet? Oh, you have to take your son to see it.”

Grace is the mother of a 43 year-old and retired from some job that required her to sit in front of a computer all day. “I already spent a lot of my life in front of a screen. I don’t want to do that anymore.” They don’t have email addresses and don’t bother with the Internet. They do walk by the ocean everyday, each one carrying a soft ball to squeeze. Grace has a red ball she kept turning in her gloved hands. Eugenio’s is a faded dark turquoise-y blue. “The hands of an artist require dexterity” he told me, fingers flexing. They laughed when I pulled out my iPhone to take notes so I would remember the names, the words, etc. and agreed that I could take their picture for this story but didn’t care to see it.

At some point early in our chat I became aware that I wanted to blog about my encounter with this couple. This awareness immediately transformed my thinking. I found myself struggling to just listen to their words once I started searching for a story I could later write. I prefer listening to, over listening for when I meet new people. It feels more organic, more natural. It also feels hard to stay present when my mind wants to narrate.

Thanks to a conversation with Steve, I got curious about the art of inquiry and how we have different perceptions of conversational openings. Thanks to Eugenio (and my love of talking to strangers) I got curious about the local work of an artist I admire for his love of the ephemeral. People (and conversations with them) are fleeting opportunities to pique curiosity and learn something new. I guess if any tip emerged from this interaction it would simply be to stay curious. And look for learning.

And that’s what art’s about, isn’t it? … It makes you see things in a different way than you would normally. – Andy Goldsworthy

Tips to Improve Your Interviewing Skills (and a request for more!)

I’m working on some of the final chapters of my book about interviewing and am interested in the ways that people have developed their own skills as an interviewer. I’ll list a few but this list can only get better with your input.

  • Practice, man, practice. It’s how you get to Carnegie Hall and it’s how you get better at interviewing.
  • Create your own practice occasions: that chatty seat mate on an airplane, the extroverted cashier – ask them a question and then ask them a follow up questions!
  • Reflect, just like a football coach who reviews the game films; watch your videos, read your transcripts, and look at what worked well and what you might have improved
  • Be interviewed whether it’s for a survey or a usability study or a poll, experience the interview from the other side of the lens
  • Critique the interviews of others (without resorting to your just-got-your-drivers’-license-know-it-all we all were at 16)
  • Observe others at work including great interviewers and poor interviewers – this can be in your work context, or in the media (Marc Maron, Charlie Rose, Terry Gross, and others) 
  • Collect war stories (more on this coming very soon)
  • Try improv 

That’s my starter list, but what have you done to get better as an interviewer?

Steve appears in two podcasts

Steve was interviewed about interviewing users by Gerry Gaffney for the User Experience podcast (transcript here, audio here) and by Jared Spool for the UIE Spoolcast (transcript here, audio here).

Listen to Steve speak with Jared Spool about “Immersive Field Research”

I was recently interviewed by Jared Spool in anticipation of November’s User Interface 16 conference (where I’ll be leading a full-day workshop).

You can listen to the interview below, and read the transcript here.

To download the audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac).

There’s a great sort of, what I think of as a great myth around that that maybe we can talk about for a sec. Just the oft repeated idea that you can’t ask people what they want because they can’t tell you so that if you’re in the kind of business and design challenge that we’re talking about where you want to break through and innovate and reinvent something you shouldn’t ask people what they want because they can only talk about what is going on today.

I love hearing that because I feel like I have a good response to that. It’s a conflation of a few things. One is, let’s just say, looking more largely, doing field research to learn about people and asking people what they want. I think if this is not an area that you’re experienced in you think those are the same thing. You think the only thing you can do in field work is to say, “well what do you want?” and then go off and build it.

And most people would say that’s not an effective technique for learning new things. I agree with that on the face of it. If you, you know, are trying to change the game in a certain space that’s well entrenched you’d better have a more interesting approach to the field than to say, “well what would you want to see different?”

You have to be looking more broadly at people’s behaviors and their needs and, you know, what are kind of educated people trying to do and how are people solving problems? What are the entrenched kind of challenges there? And so you need to use techniques to gather that information and make sense of that information.

It’s not a ladle that you dip into the soup, right? Scoop. Oh, here’s what people said they want. We’re going to go off and do it. That’s never a way to do breakthrough stuff. So yeah, I agree when people say don’t ask what people want because they can’t tell you but I don’t agree with the implication of that which is don’t do research to try to innovate.

Flow in the interview

Earlier this week the San Francisco IxDa hosted a talk by Peter Stahl about The Rhythm of Interaction. As part of his presentation , Peter talked about Mih?°ly Cs??kszentmih?°lyi’s notion of Flow – “the mental state of operation in which a person in an activity is fully immersed in a feeling of energized focus, full involvement, and success in the process of the activity.”

Yesterday I came across a podcast I did a few years ago with the folks from Lunar where we talked about how speed, creativity and innovation intertwine in the design process and about getting results through design research. You can listen to the podcast at the bottom of the post; meanwhile I’ve pulled out a snippet where I describe entering a flow state when interviewing users.

And all the power of noticing and stepping back and slowing yourself down and just disengaging yourself from the need to be making things happen, is just sort of creating that space and t hat’s where insights happen. That’s where creativity can happen. And I’m sure you guys have seen that moment when you’re in the field, where you have all this responsibility to be managing a session and managing the other people in the session and making sure you stick to your time, and it’s a lot of, lot of work. Your brain is just firing on all its cylinders. And then sometimes for me there’s that moment where you kind of – it’s almost like a hyperspace moment where the starts start to just stretch out. Things just get really, really quiet in my head and suddenly, I’m just riding it. Things are sort of happening and I’m riding it, and that can be – it’s, I guess, a flow moment, right? Things can be really insightful at that moment. I don’t know that I’m bored, but if I had to contrast that to the stimulation of trying to run everything and run everybody, that seems to be a really kind of creative moment for me when that happens.

Listen to the podcast:

To download the audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac).

Win Your Subjects Over with Genuine Enthusiasm

Parallels to interviewing users crop up all over the place. Just like us, portrait photographers ask their subjects trust them, as they go on a journey that might be uncomfortable at times. I don’t want to overwork the comparison, of course. You’ll see more differences than similarities in this video, but what struck me was the core notion that your own sincere enthusiasm will serve to build that trust.

Chris Buck promotional video

(via PetaPixel)

From SXSW – Diving Deep: Best Practices For Interviewing Users

I spoke at SXSW recently about the interviewing process in user research. As I’ve been working on a book about this topic the timing was great for me.

While we know, from a very young age, how to ask questions, the skill of getting the right information from users is surprisingly complex and nuanced. This session will focus on getting past the obvious shallow information into the deeper, more subtle, yet crucial, insights. If you are going to the effort to meet with users in order to improve your designs, it’s essential that you know how to get the best information and not leave insights behind. Being great in “field work” involves understanding and accepting your interviewee’s world view, and being open to what they need to tell you (in addition to what you already know you want to learn). We’ll focus on the importance of rapport-building and listening and look at techniques for both. We will review different types of questions, and why you need to have a range of question types. This session will explore other contextual research methods that can be built on top of interviewing in a seamless way. We’ll also suggest practice exercises for improving your own interviewing skills and how to engage others in your organization successfully in the interviewing experience.

Slides



Audio

Presentation (45 minutes + 15 minutes Q&A):

To download the audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac).

Also see

Unfinished Business lecture: Culture, User Research & Design

I was recently in Toronto to speak at OCAD (Yes, we were in this awesome building) as part of the Unfinished Business lecture series. My talk looked at the notion of culture and it’s importance for user research, and design.

Culture is everywhere we look, and (perhaps more importantly) everywhere we don’t look. It informs our work, our purchases, our usage, our expectations, our comfort, and our communications. In this presentation, Steve will explore the ways we can experience, observe, and understand diverse cultures to foster successful collaborations, usable products, and desirable experiences.

Slides



Audio

I’ve split out the presentation itself from the Q&A, which was fun, challenging, and filled with big-picture type questions.

Presentation (1 hour, including a quick intro by host Michael Dila):

Q&A (40 minutes):

To download the presentation audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac). For the Q&A audio, Right-Click and Save As… (Windows) or Ctrl-Click (Mac)

Note: In the talk (and the Q&A) I refer to my interactions article, Persona Non Grata. You can find that article here.

Our latest article: What to Expect When You’re Not Expecting It


Our latest interactions column (written in collaboration with Julie Norvaisas) What to Expect When You’re Not Expecting It has just been published.

We are inevitably astounded and affected by what exists outside of explicit research project constraints.We indulged in a little reflection on some of the people we’ve met and how meeting them took us outside of the business questions at hand but had a real impact on the team and reframed the way we thought about the business questions. This opportunity to dwell on the exception provides a reminder of how these experiences deliver a potent dose of humanity to the business of providing products for people.

Get the PDF here.

Previous articles also available:

Easy Listening

[Note: I was asked by a national print publication to join their crowded roster of design bloggers; Over a few months we worked together on my pitch and eventually I wrote and shared my first post. They were quite keen and ran me through all the technical and style guidelines for using their site. But then they asked me at the last minute to hold as they relaunched their blog. Then, silence. The discussion of my series fell down a hole. Given that almost a year has gone by, I’ve realized that it ain’t happening anytime soon. So here’s the piece!]

Rahul turned to Amanda, his eyes sparkling with excitement. “Hey, I saw a very strange dog today. You wouldn’t believe it!”
Amanda placed a finger in her novel and looked up. “What?”
“A strange dog. I saw a strange dog today.”
“Oh yeah-?” Amanda trailed off, her eyes dipping back to her book.

This is how we live today (I’m not saying it was always this way; did loquacious primitive Thag grunt enthusiastically while Klag scratched drawings upon the cave wall?). Sometimes we’re distracted, busy, tired, or just not that interested. Hearing these stories takes energy (isn’t that right, introverts?) or maybe we’d rather share our own story (isn’t that right, extroverts?). Even when we do engage in conversation, we’re often thinking about what we want to say next, and listening for those breathing cues that indicate it’s our turn to speak. Listening is a limited resource. No wonder we pay people to listen to us talk about ourselves!

And while companies acknowledge the value of listening to customers (what new feature, good or bad, isn’t announced without mealy-mouthed PR justification that “We listened to our customers and they told us-“), even at best that’s often just lip-service. As an individual skill that is crucial is so many business interactions, it’s woefully underdeveloped. While we’d all likely check off “good listener” on a self-assessment, it’s something we should probably get better at.

We don’t have the space (nor the qualifications) to help you get to a point where you care about what your client, customer, colleague or loved one has to say, so let’s just take that as read. But once you’re in the conversation, how do you stay in? One tactic involves your body.

Maybe you’ve heard the phrase “act as if” from the worlds of life coaching, personal growth, or therapy (i.e., acting as if you aren’t anxious is a tool for dealing with anxiety). By the same token, if we act as if we are listening, we’ll find it easier to listen.


The body language of good listening


Not so much

In The Naked Face Malcolm Gladwell describes the work of psychologists who developed a coding system for facial expressions. As they identified the muscle groups and what different combinations signified, they realized that in moving those muscles, they were inducing the actual feelings. He writes

Emotion doesn’t just go from the inside out. It goes from the outside in-In the facial-feedback system, an expression you do not even know that you have can create an emotion you did not choose to feel.

It’s a likely extension of this finding to imagine bodily expressions that demonstrate emotion and intent similarly creating those matching feelings in us. Even if it isn’t true, these postures send strong signals to our interlocutor, further encouraging them to share with us.

One of my favorite ways to practice listening is via serendipitous encounters with loquacious taxi drivers, airplane neighbors, or social-cue-missing party chatters. Even if we can’t repair society’s listening inequity, we can use it to provide endless practice space.

For more about listening, you should check out

Recap of Steve and Julie’s URF10 synthesis workshop

Our friends at Bolt | Peters hosted their (mostly) annual User Research Friday event last week, bringing together practitioners from the client-side as well as consultants to share stories and discuss best practices. Some of our takeaways from the day are here.

The day before the conference, Steve and Julie co-led a sold-out workshop titled “We’ve Done All This Research- Now What?” for a group of 20 enthusiastic researchers and designers.


Julie and Steve in action

The purpose of the workshop was to practice the process of moving from the data and observations we gather in fieldwork toward opportunities and ultimately to ideas.

We framed this as a research project to inform a neighborhood redevelopment/gentrification effort. Before the workshop, participants first wandered their own neighborhoods…


Thanks to Nick Leggett from Zazz for this aerial shot from their Seattle offices


Noe Valley scene (a San Francisco neighborhood) captured by Julie

…and then when we got together, they the explored neighborhood surrounding Bolt | Peters for more data.


This machine shop just down the street from Bolt | Peters has been there for decades


6th street buzzes, about two blocks from the conference

Break-out groups took the synthesis tasks to heart and, in a very short period of time, collaboratively surfaced promising opportunities and strategies and solutions to address them.

We were humbled by the gentle empathy and creativity of the folks in the room. The morning served as an inspiring reminder of just how much progress a handful of smart, dedicated people can make on seemingly-intractable problems in a very short period of time.

More amazing photos, observations, output, and thoughtful commentary can be seen on the blog we created for the workshop.

The workshop slides are below.

See previously: Steve Portigal’s presentation from User Research Friday 2008

Local norms for listening vs. telling


Fieldwork transcripts are divvied up among team members

Over a few days spent reviewing interview transcripts from the US and China, I was struck by the observations around storytelling vs. listening vs. followup questions in Returning to America from a life in China (abstract only)

[I]n the States, I often had trouble responding to personal stories. But soon I realized that it didn’t make much difference what I said. Many Americans were great talkers, but they didn’t like to listen. If I told somebody in a small town that I had lived overseas for fifteen years, the initial response was invariably the same: “Were you in the military?” After that, people had few questions. Leslie and I learned that the most effective way to kill our end of a conversation was to say that we were writers who had lived in China for more than a decade.

At the times, the lack of curiosity depressed me. I remembered all those questions in China, where even uneducated people wanted to hear something about the outside world, and I wondered why Americans weren’t the same. But it was also true that many Chinese had impressed me as virtually uninterested in themselves or their communities. That was one of the main contrasts with Americans, who constantly created stories about themselves and the places where they lived. In a small town, people asked very little of an outsider – really, all you had to do was listen.

Mike Tyson and the Power of Holding Your Tongue

The 2008 documentary Tyson by James Toback is a compelling and revealing work. From a technical perspective, it’s a fun watch because Toback experiments with visual fragmenting and layered storytelling styles. In terms of subject matter, one would be hard-pressed to find a juicier, more tabloid-soaked figure to focus on, especially for those of us who came of age in the 80s. I walked away from the film with a much more nuanced and complex, though still ambivalent, view of Mike Tyson as a powerhouse boxer, as a convergent cultural figure, and, finally, as a very complicated human being.

But there was one moment that stood out, and it hammered home the incredible power of a simple interviewing technique: silence. At one point about mid-way through the film, Tyson was yammering in a very straightforward way about the fact that his desire to box and dominate stemmed from his being bullied as a young boy (predictable!). Toback must have sensed something simmering just below the surface, because when Tyson finished this train of thought Toback just let it sit. And sit. And sit. As the audience sits. And sits. Until Tyson looks back up with a completely different expression, almost with a different personality, and bares the real, brutal truth. It’s a moment when time kind of stops; I gasped out loud. It’s this kind of thrilling moment that we experience in our best interviews, when the person (“consumer!”) goes beyond just citing facts or recounting stories, to communicating to us, and our clients, something surprising, something of real value and meaning.

If you liked this interview tip, you’ll love this: Steve will be talking about his interviewing secrets at the UIE virtual seminar on the 28th of this month!

Don Norman says design research is great for improvement but useless for innovation

Don Norman, in a sneak preview to an upcoming column in interactions, posts a dramatic and thoughtful critique of the supposed applications for design research

I’ve come to a disconcerting conclusion: design research is great when it comes to improving existing product categories but essentially useless when it comes to new, innovative breakthroughs. I reached this conclusion through examination of a range of product innovations, most especially looking at those major conceptual breakthroughs that have had huge impact upon society as well as the more common, mundane small, continual improvements. Call one conceptual breakthrough, the other incremental. Although we would prefer to believe that conceptual breakthroughs occur because of a detailed consideration of human needs, especially fundamental but unspoken hidden needs so beloved by the design research community, the fact is that it simply doesn’t happen.

I’m excited to see this because it connects to a number of things I’ve been talking about with clients and in some recent presentations. Anyway, the article makes some good points but I believe there’s much more to be said.

  • Absence of evidence is not evidence of absence – Design/design thinking/design research are in their infancy in product development. The airplane, automobile, telephone, etc. are not examples of the failures of design research to innovate, because they represent time periods when design research was not actively being used. As Don points out, the failure rate for potentially innovative stuff is insanely high. So we have very few examples over a few few years to even look at to understand the influence of design research.
  • Innovation is not a solo act – There’s probably a good Andrew Hargadon link I could add here, but I think you get it. We point our client to opportunity areas. Many of those opportunities do not get fully explored, and almost none to the point of solving the ridiculously challenging technical and business challenges to make them viable. The Conversation was potentially a breakthrough film not only because of Coppola (a successful innovator) but because of Hackman and Murch. And many other talented people. When our design research leads to a divergent set of concepts, other factors come into play. The remote-activated-deodorant-ray (yes, this came out of an actual client project) goes through the design team, the business unit manager, eventually into the technology development part of the business, and the market feasibility. Most times that doesn’t happen. And maybe this just makes Don’s point for him, but then I’d suggest the problem is not with design research but in how it’s deployed, applied, and integrated. Because it absolutely could happen. The underlying conditions need to be there.
  • Can insight and technology be partners? – There are presumably a number of paths to innovation. If we uncover opportunities through design research, a technologist can say “Well, let me go try and make that” (or, “I’ve already figured out how to do that”). Or if a technologist approaches us with a set of capabilities, we can try to answer the question “What would people do with it?” Again, maybe I’m making Don’s point for him, but if so, I don’t see it as a negative.
  • Isn’t this still a mostly mysterious process? – Twitter is a successful product with a low barrier to usage but a high barrier to adoption. It’s success is somewhat counter-intuitive. The traditional market-research processes that failed the Aeron chair and the Post-It note are already consultant-classics. Maybe I’m admitting something terrible but I don’t think Tim Brown or Larry Keeley or Roger Martin can identify the next breakthrough product any more than Hollywood can figure out the best way to guarantee a blockbuster or the recording industry can sign the next number-one band (indeed, look at the amount of marketing hype and me-too that goes into the product development approaches of the last two).
  • Innovative (if that’s what they are) outcomes take years to launch – I’ve written about this before. Maybe what I’m calling innovations are really what Don calls improvements. But I don’t expect ever to contribute to the next Telephone/Airplane/Computer, but I don’t expect to be President of the US, or win an Academy Award, or have one of my songs hit number 1. That doesn’t mean the work isn’t worth doing and the results can’t be tremendously successful, impactful, and result in real change.

I think Don has written a thought-provoking piece and my intent is to reframe rather than refute. This is an important discussion that needs to continue and I am eager to see what others have to say. If you’ve written about this, please post a link here!

5 Keys to Successful Design Research

hack2work
Our friends at Core77 have launched Hack2Work: Essential Tips for the Design Professional.

The feature includes a generous serving of amazing content including pieces by Tim Brown, Alissa Walker, Michael Bierut, and Liz Danzico. My contribution is 5 Keys to Successful Design Research

1. Embrace your participants’ world view
Great research will help you understand how the people you are designing for organize and describe the world. Their words reveal their frame of mind. That means you must discover your own jargon, and let it go. Just because you are designing a netbook doesn’t mean that your research participants will view it as anything other than a “tiny laptop.”
[more]

Check it out!

Previously: Hack2School: Practice noticing stuff and telling stories

Breathe their air

algebraist

In Iain M. Banks’ “The Algebraist”, the protagonist Fassin Taak is a “Slow Seer” who spends years embedded alien cultures (including the complex Dwellers), engaging in conversation and seeking insight, in an activity referred to as “delving.” Because he favors traveling to these other planets in person, he is challenged by the establishment who prefer more efficient methods [boldface emphasis mine].

“Have you tried remote delving recently?”
“Not for a long time,” Fassin admitted.
“it’s changed,” Pagges said, nodding. “It’s much more lifelike, if you know what I mean; more convincing.” Paggs smiled. “There have been a lot of improvements over the past couple of centuries.”

Ganscerel patted his arm again. “Just try it, will you, Fassin? Will you do that for me?”
Fassin didn’t want to say yes immediately. This is all beside the point, he thought. Even if I didn’t know there was a potential thread to Third Fury, the argument that matters is that the Dwellers we need to talk to just won’t take us seriously if we turn up in remotes. It’s about respect, about us taking risks, sharing their world with them, really being there.

In Seventeen ways to not suck at research, number 9 was Breathe their air, my response to the increasing desire for remote ethnographic-like methods. I think the insight and empathy that is gained from getting out of our own environment is essential, and as Banks points out, showing up on someone else’s planet is a a very effective way to start building rapport.

(Ironic detail given my choice of metaphor: Dwellers reside in/on gaseous planets, so Fassin actually visits them encased in a pressurized vehicle, and never literally breathes their air).

Also see Great interviewing means feeling the subtext

Great interviewing means feeling the subtext


I was inspired when I came across this bit from John le Carré’s “A Most Wanted Man”

And again Bachmann let the observation pass. A halfway-decent interrogator, he liked to preach…doesn’t smash the front door down. He rings the front doorbell, then goes in at the back entrance. But this was not the reason he held off, as he later confessed to Erna Frey. It was the other music that he was hearing: the feeling that, while she was telling him one story, he was listening to a different one, and so was she.

A number of great interview principles here (can you think of any others?)

  • Ease your way along
  • Pay attention to your feelings about what information is lurking ahead
  • Don’t force subtext to become text

This is not a process of ask-question-collect-answer-repeat. It’s filled with wonderful subtleties and nuances. Bachmann, le Carré’s interrogator, has a good handle on them as they apply to his context.

Speed of Innovation: Steve Portigal featured in Lunar Design podcast

Inspired by my interactions column Hold Your Horses, I was interviewed for a Lunar Design podcast.

How do speed, creativity and innovation intertwine in the design process? In this Connections episode, Gretchen Anderson and Lisa Leckie talk with Steve Portigal of Portigal Consulting about getting results through design research.

Listen to podcast:

User Research Friday: Research and Design, Ships In the Night? (Updated)

(Updated to include audio, video, and interactions article)

Here are my User Research Friday slides, along with audio and video. For me, the discussion at the end (there was a bit of stunned first-talk-of-the-day silence during question period so I turned it around on the audience and asked them to comment on the Escher-esque slide about design->research->design->) was the most stimulating part.



Listen to audio:

I’ve turned this talk into a two-part column for interactions; Get a PDF of the first part here here and the second part here.

Listening vs. Hearing

In Fast Company’s Green Guru Gone Wrong there’s a sobering examination of sustainability architect William McDonough and the work that he’s doing. I am sure this type of investigation is highly contentious, especially when icons like McDonough are revealed to be less-than-perfect.

But it’s interesting to note that some the project failures are tied to a dramatic lack of understanding of the current behaviors and future needs of target customers.

Shannon May smelled the rot firsthand. An anthropology PhD student from UC Berkeley who lived in Huangbaiyu for nearly two years, May first met McDonough in 2005, the year the project broke ground. But within several months, it became apparent to May that everything from the village’s overall design to its construction was deeply flawed. The homes were suburban-tract style with garages, despite the fact that only four of the expected 1,400 villagers had cars. The backyards were too small for growing feed corn or raising animals, which the villagers needed to make their living. But most absurd to her eye was the plan to use agricultural waste to fuel the biogas plant to power the village: leftover corncobs and stalks were the winter food supply for the cashmere goats, the area’s leading source of cash. Using them meant the goats would starve.

“I started calling Bill and telling him these things, and he would be very responsive and concerned on the phone,” says May, the blonde seen standing behind McDonough in Friedman’s documentary. “What troubled me was that it was as if he knew nothing about the way these people lived. And he seemed concerned, but then nothing would happen after these phone calls.” May says McDonough visited the village only twice while she lived there “for one or two hours at a time, and only when there was a video camera following him.” The supposedly $3,500 homes were costing nearly $12,000 to build, more than 10 times the villagers’ median income. By 2006, only two families had moved in, and they did so because their previous homes had burned down. Even then, they had to use antiquated heating rigs because the renewable energy systems didn’t work.

And even more interesting is that the failure isn’t about a lack of information about these customers, it’s a failure of process to integrate that information into the project decisions.

Please wait here. We’ll be right back with some fresh hot insight.

I was interviewed recently for What insight is ethnography delivering? (PDF). It’s a pretty clear piece, and I think we show well in it. Lots of tidbits but the closest thing to controversy is this:

Portigal accepts that while there needs to be conditions, such as a ban on any logos being worn by the accompanying client, and an agreement to undertake some basic workshop training to introduce them to the principles of field work, he is happy to bring along a member of the client’s team. “It’s the 80/20 rule: we ask 80% questions, you ask 20%. It takes them a couple of practices and then I think they can make a really valuable contribution.”

However, for O’Brien, the very thought of having a non-team member accompany out in the field is a non-issue. “We simply don’t believe in it,” he says. “The fewer people, the better. If you start crowding a room out, how is the participant going to feel comfortable? In fact, we have even lost jobs over it.”

It took me many years to come around to this way of thinking, but as our work has become more about facilitating our clients to take action and less about handing off insights, it seems right on for us. I’d love to hear what you think!

Experienced pollsters know: people “lie”

As I’ve said before, garbage in, garbage out. From Rob Walker’s Consumed

Recently, Stardoll did a study of its own, polling United States users about their brand preferences. Apparently they saw real-world brands on the same plane as the half-dozen or so invented brands that exist only within the site. (Some respondents even made the – clearly impossible – claim that they wear the strictly digital Goth-style brand Fallen Angel to school.)

These sorts of stories always crop up in market research and business case studies. And they are wonderful because they illustrate the depth of meaning the products, services, brands, and stories we create can be to the people that consume them. So meaningful that they will conflate pretend brands online and tangible experiences offline. Wow, we marvel, that tells you how great our stuff is; they will lie about it.

But the flip side to that is that if you are going to ask people what they think and do and want, you better have a way of triangulating their responses against other data. If you don’t know more about the person than their response, how can you contextualize it? If you don’t know what they are really saying when they answer the question – if they understood the question or are answering it in the way you intended – then you must be very careful in what you conclude and how you act on those answers.

Peeling The Onion

From our Design Research Methods class, some observations from an interviewing exercise.

The scenario was to conduct interviews in order to uncover opportunities in helping people to manage food, meals, nutrition, etc.

The question asked was
What are the challenges you face in meal preparation?

Of course, that question is flawed because it presumes that there are indeed challenges. This was evident when the respondent struggled with how to answer outside the frame of the question.

An alternative might be
Are there any challenges in meal preparation?
which is is more open-ended.

But better still is
What are your feelings about the experience of meal-preparation?
since it doesn’t put the label challenges into play. It would be important to understand the labels the person being interviewed places on the different aspects of their experience, and to use their terminology to probe further.

Also worth noting is that the original question came right off the sample interview guide I distributed. Sometimes the interview guide is a tool to document “questions you want answers to” rather than “questions you want to ask”; doing fieldwork involves a lot of translation back and forth between the two.

More on noticing and reflecting

Recently I wrote about the importance of practicing our noticing and reflecting skills. A few weeks ago I read on MetaFilter about John Stilgoe “a professor at Harvard who teaches his students how to, among other things, mindfully observe the urban and suburban environments they inhabit.”

I bought his book Outside Lies Magic, and although the book itself is so-so, the introduction is passionately articulate about some of these same issues I’ve written about

It is a book about awareness in ordinary surroundings. It is a book about awareness that builds into mindfulness, into the enduring pleasures of noticing and thinking about what one notices.

I hope this book encourages each reader to widen his or her angle of vision, to step sideways and look at something seemingly familiar, to walk a few paces and see something utterly new.

Interviewing and Soundbites

Yesterday on KQED’s Forum their guest was Brooke Gladstone from On the Media. I was struck by this quote, early in the broadcast.

I will ask a question…maybe 5, 6 times, and the person we’re interviewing gets around to answering bits and pieces of it after every iteration and we will sometimes assemble all those answers into one.

This is the art of the interviewer — probing, following up, asking again, in order to get to the answer, eventually. It’s why interviewing in teams is tricky, because the other member of the team needs to know what’s in your head, to give you space to ask that question over and over again til you get to it. Regular people do not speak in soundbites.

We also see this over and over again when we’ve synthesized conclusions and we go look for supporting video data to share with our clients. It’s just not there as some obvious artifact. That’s why, of course, our recommendations come from synthesis — insights are not sitting there waiting to simply be scooped up, they have to be assembled from a variety of sources.

I am envious of those with amazing audio/video editing chops who can use the data to more closely approximate what we want to say; but at the end of the day, the big ideas rarely come out of people’s mouths directly.

Research screening

I was bemused to see that Feast of Love opened last weekend. Our last time at the movies was when The Simpsons Movie opened, and I participated in some intercept-market research at the theater.

Part of the lobby had been given over to these groovy looking kiosks, with a couple of guys in attendance, asking people who passed by if they would like to give their opinion about an upcoming movie. My age and gender qualified me to participate (woo hoo) and I went with one dude over to a kiosk. I was shown a couple of clips and responded to various questions, but the weirdness of it was that the test was designed have some screens operated by me, and some screens visible only to the interviewer. But they didn’t do it that way. So for various pieces where I was to click within multiple choices, the interviewer, who knew the testing software rather well, just whipped through the keypresses, bam->redraw, bam->redraw, quickly asking me the minimum to move to the next one. Okay, so he took care of it for me. But then this screen we were both looking at would display testing instructions such as ASK PARTICIPANT FOR OPINION OF BENEFIT OF DATE MOVIES. PROBE ON RELATIONSHIP, TIMING, COST. And of course, he wouldn’t even come close, he’d get the one line answer from me, and then he’d type in the quickest condensation of my answer: stay home.

After a minute or so, it became more about the two of us cooperating to use the software to get through test. I realized that my opinion didn’t matter; it’s hard to feel represented in a forced-choice discussion, and it’s unlikely one would continue to provide color when all that gets captured is minimal facts. Further, by exposing the instructions to me, his shortcuts became clear, and I ended up slightly co-opted into the testing process, giving up any sense of really delivering the full truth to this interviewer.

When we see “market research” number published to support some business decision, let’s keep in mind how poorly that data may have been collected (from the concept of how to collect that data, to the implementation of a data collection environment, to the staffing and execution of the data gathering). How reliable could any of this possibly be?

Seventeen ways to not suck at research


I spoke recently at Shift, the IDSA conference held at RISD. I outlined seventeen ways to not suck at research:

1. Quit worrying about jargon
2. Think more broadly about which people you want to learn about
3. Garbage in, garbage out
4. Give other people the space to tell their stories
5. Follow up, and then follow up, and then follow up
6. Do you really want to use a survey? Probably not.
7. Collect and use natural language
8. Don’t forget that any research process with real live humans is hard
9. Breathe their air
10. Learning anything new requires rapport, and building rapport takes time
11. Finding insights requires pattern matching, creativity, synthesis
12. Personas are user-centered bullshit
13. Phil McKinney says “You’re probably not listening.”
14. Practice noticing stuff and telling stories (updated: read more here)
15. Do some improv
16. Embrace pop culture
17. Don’t forget about culture and social norms

The presentation was very well received, and I hope to share this material with another group before too long.

img_0232.jpg
img_0231.jpg

Tell me how you.. vs. show me how you…

We’re doing a bunch of fieldwork these days looking at how people are using their software and hardware to accomplish some tasks. It’s interesting to see the difference in the flow between the part where we ask people to describe what they do and how they do it, and the part where we ask them to show us how they do it.

The discussion part is hard. It’s a bit abstract to explain a detailed behavior in absence of any props or artifacts. People work to give clear explanations and it takes a lot of follow-up to get the details.

Most of the same people light up when they are asked to show us (although some simply decide to show us without prompting).

But I find myself liking the “tell me how” part of the interviews better; the comfort level is lower, but the struggle to articulate is very insightful. Looking at how people describe things from memory isn’t wholly accurate in capturing their perceptions or usage, but it pulls out some neat contrasts.

And ideally, we’re trying to get stories, not factoids. The discussion (not the demo) is much better for stories. The participant takes over in the demo, and it becomes a semi-hurried list of “this works like this, and this other thing works like that.”

Of course, we’re doing both, and we need to be doing both, and some of the insights will come of the tension or differences between the two.

I think we’ll switch the order on today’s interviews, and maybe try starting with the demo, and then doing a discussion afterwards. I sometimes feel the demo requires time, and rapport, and trust, before we can safely ask for it (especially if the equipment in question rests within an inner sanctum), and so this is a bit of a leap for me.

Of course, there are few “right” answers in evolving one’s technique, it’s about building up a larger palette of approaches and making intelligent choices about when to switch around. I’m not at all unhappy about how the interviews have gone so far; they’ve been fun and fascinating, but I’m thinking hard about how to keep doing better.

Global Innovation Research

Recently we were interviewed for two different global studies on global innovation. Both the German Research Foundation or Deutsche Forschungsgemeinschaft and the Danish Ministry of Economic and Business Affairs and Centre for Economic and Business Research or FORA separately came through the Bay Area as part of as study of leading innovation/design entities in the US and around the world.

Both groups offered to share their results with me and so I’ll post more specifics here when I’ve got ’em. Meanwhile, it was fun to be interviewed instead of doing the interviewing, even if it was clearly a conversational, expert-interview type of thing, and not an ethnography (neither did a huge amount of following-up or probing, it was pretty much based on what I had to say). One interesting technique, though: the group from FORA asked me to reiterate some of my key points from our longer interview (recorded on audio) for a 5-minute summary video. They went over with each other, and with me, what points seemed the most salient, and then reinterviewed me into the video camera. The point was not to get more information, but simply to create an effective deliverable. I was not at all uncomfortable with it, indeed I felt very listened to in having a precis of my interview created on-the-fly. It reminded me that being able to communicate what is learned from the field is crucial; that methods could be innovated around the effective communication as much as the effective discovery.

Wal-Mart dramatically retargets (ahem) based on user research

In today’s New York Times

There are “brand aspirationals” (people with low incomes who are obsessed with names like KitchenAid), “price-sensitive affluents” (wealthier shoppers who love deals), and “value-price shoppers” (who like low prices and cannot afford much more).

The new categories are significant because for the first time, Wal-Mart thinks it finally understands not just how people shop at its stores, but why they shop the way they do.

Of course those segment labels are dehumanizing and unpleasant, but the source for this new understanding, years of in-depth studies with customers, must have been some very insightful research. Congrats to friends who I presume were the ones that actually did that research (even though they’ve moved on from Wal-Mart)!

Update: The friends disclaim any credit for this work!
Update2: This leaked PPT presumably explains their methodology.
7e05cff.jpg

genchi genbutsu – that’s Toyota for “user research”

There’s tons of good stuff on business/manufacturing/processes/marketing/company culture/innovation in the fantastic article about Toyota from the Sunday NYT magazine. I’ve picked just a bit to share here.

Toyota’s chief engineers consider it their responsibility to begin a design (or a redesign) by going out and seeing for themselves – the term within Toyota is genchi genbutsu – what customers want in a car or a truck and how any current versions come up short. This quest can sometimes seem Arthurian, with chief engineers leading lonely and gallant expeditions in an attempt to figure out how to beat the competition. Most extreme, perhaps, was the task Yuji Yokoya set for himself when he was asked to redesign the Sienna minivan. He decided he would drive the Sienna (and other minivans) in every American state, every Canadian province and most of Mexico. Yokoya at one point decided to visit a tiny and remote Canadian town, Rankin Inlet, in Nunavut, near the Arctic Circle. He flew there in a small plane, borrowed a minivan from a Rankin Inlet taxi driver and drove around for a few minutes (there were very few roads). The point of all this to and fro, Jeff Liker says, was to test different vans – on ice, in wind, on highways and city streets – and make Toyota’s superior. Curiously, even when his three-year, 53,000-mile journey was finished, Yokoya could not stop. One person at Toyota told me he bumped into him at a hotel in the middle of Death Valley, Calif., after the new Sienna came out in 2004. Apparently, Yokoya wanted to see how his redesigned van was handling in the desert.

and

The way a farmer uses a truck is different from the way a construction worker does; preferences in Texas (for two-wheel drive) differ from those in Montana (for four-wheel drive). Truck drivers have diverse needs in terms of horsepower and torque, since they carry different payloads on different terrain. They also have variable needs when it comes to cab size (seating between two and five people) and fuel economy (depending on the length of a commute). In August 2002, Obu and his team began visiting different regions of the U.S.; they went to logging camps, horse farms, factories and construction sites to meet with truck owners. By asking them face to face about their needs, Obu and Schrage sought to understand preferences for towing capacity and power; by silently observing them at work, they learned things about the ideal placement of the gear shifter, for instance, or that the door handle and radio knobs should be extra large, because pickup owners often wear work gloves all day. When the team discerned that the pickup has now evolved into a kind of mobile office for many contractors, the engineers sought to create a space for a laptop and hanging files next to the driver. Finally, they made archaeological visits to truck graveyards in Michigan, where they poked around the rusting hulks of pickups and saw what parts had lasted. With so many retired trucks in one place, they also gained a better sense of how trucks had evolved over the past 30 years – becoming larger, more varied, more luxurious – and where they might go next.

Obu’s team, which drew on hundreds of engineers, ultimately produced a pickup model with 31 variations that include engines, wheelbases and cabs of different sizes. Design engineers, however, cannot simply create the best truck they can; they need to create the best truck that can be built in a big factory. In other words, Tundra’s design engineers had to confer with Tundra’s manufacturing engineers at every step of the way to create a truck – or 31 trucks, really – that could be assembled efficiently and systematically.

This is what happens in user research!

Snipped from an article today about chemical-based cleaners in the home

Cory McKee, 27, a stay-at-home mom of three in Tridell, Utah, started ordering Seventh Generation brand cleaning products online two years ago after learning that her oldest child, now 7, had celiac disease, a gluten intolerance. Ms. McKee said that although the disease is not caused by toxins in the home, dealing with it raised her awareness about other health issues. [italics mine]

“That really woke me up,” Ms. McKee said. “I really need to make sure our home is safe.” She lost confidence in the cleaners she had been using in part because the labels of some products do not list all of their ingredients. That made it impossible to know what her family was being exposed to when she sprayed the windows, she said.

I love this, even after it’s gone through the journalistic clarifying filter. People’s ideas jump from one arena to the next. We conflate different concerns. Ask someone about their eating habits, and they’ll talk about exercise, or ask about being fit and they’ll talk about bedtime, or how to stay calm and deal with stress. We’re not good at compartmentalizing. And so ethnographers, using a conversational tool, encounter this all the time. A decision about one thing is related to a concern about something else. Even though there was no causal relationship between the celiac disease and the cleaners (and not to mention that they child already has celiac disease) the person being interviewed puts forth a causal relationship.

I hear this sort of thing all the time when I talk with people, and it’s usually much less clear than this. Complex purchase decisions, tasks, and lifestyle choices (of which our lives are full) often are told in slippery stories that start off in one place and end up in another. Teasing those apart (as this writer did here), asking for clarification, and being open to understanding how A could possibly connect to B in someone’s mind are crucial to getting at those applicable insights.

Our readers write

Charles Frith emailed a question that he hoped I could address here

Is ethnography more suited to shedding light and depth on peoples relationships with some products or services over others? Are there times when it’s completely unnecessary?

That’s a two-parter! I’ll take a shot at this all, but I hope that others will jump in to clarify, correct, disagree, or whatever.

Are there times when ethnography is completely unnecessary?! Absolutely. But your second part is about time and your first part is about category. In terms of time, in terms of the process of developing a product or service, there are different types of research that may be more or less appropriate at different points along that process. There’s no overall answer, for me, it always depends.

For example, who is the organization? What do they know about the category, or the customer, or the channel, or the market, at this point? Is it a new product in a new category, or a redesign of something they’ve been doing before that they are improving? These are all facets of “what do you need to know more about?” I guess.

The best times to do any sort of customer research (and sorry if this is obvious) is when you can take some action with the results. If there are decisions to be made and they need informing by a user/customer perspective, then try to do that research ahead of time.

Depending on how you define ethnography, you may wish to see it only as a discovery process, as something that happens early on to define needs so that you can create solutions to match those needs. But putting solutions – prototypes – back into that conversation as a way to further validate the needs or dive deeper into them is a great way to go. You can read a case study of ours here that relates how an early prototype was taken into homes in order to have a more meaningful discussion about what the needs were and how this product could address those needs. You can always hold back your artifact for some portion of your session; that’s a standard technique we use.

As far as categories of products and services go, I don’t see that as a big dividing line in terms of the usefulness of research (and I’m speaking about research in general, whatever methodology you want to consider, some way of bringing customer perspectives into your design and decision process). I’d point again to the organization and its process as the crucial gating factors.

And

Also, are there precedents for ethnographic research in Asia?

Wow – two opportunities to cite DUX conference papers in one blog post. This PDF uses research in Japan (conducted by us, and by Adobe, over several years) as a template for considering research in global locations.

I think there’s two flavors here: Global companies coming to Asia to understand these markets, and Asian companies doing for their own domestic markets something like ethnography. For the first, absolutely. Japan, China, and India have all been high-profile destinations for user research (and in case anyone wonders, we are very keen to go back and do some more). For the second, I hear more about India, specifically, with an increasing drive by Indian companies to understand how their products and services will be received. A colleague in India does a regular audit of teenager attitudes for MTV India. I don’t know in much more detail the market for this sort of thing in any part of Asia, only the different anecdotes I’ve collected from various Asian colleagues.

Should I also re-plug CultureVenture at this point? For companies looking to get a handle on these other markets, this is our service to drive inspiration through immersion. Facilitated hanging out, if you will?

How’s that Charles? I hope you’ll add some follow up questions in the comments.

My UXWeek Session

Here is a link to a podcast page for my recent talk at Adaptive Path’s UXWeek.

Effective user research requires both observation and interviewing. When doing research we strive to get outside our own default expectations and perceptions, in order to better see the details of what we’re looking at, in other words, to understand the cultural context. This third component is the most crucial to innovation. Interesting things happen when we leave our homes and our comfort zone, perhaps in another country where business, language, food, and more is beyond our own frames of reference.

Steve Portigal, founder of Portigal Consulting, offers expert tips in both observation and interviewing, and considers the challenges and opportunities in conducting research abroad. He believes that one way to better understand a different culture is to look at how things in your own culture are handled differently. He gives some examples of how some things are promoted differently in Japan than in the United States. He states that mundane observations reveal important cultural differences.

Here are the slides

I’ll be curious how others fare; I couldn’t get it to go past slide 6, although I could go to the end and page backwards. I’ve reported a bug and hope it gets fixed, but since it’s just been launched and getting some buzz, who knows. I re-uploaded and replaced it with a version that works. They are working on tracking down the problem with the last version.

NPR : The Art of the Interview, ESPN-Style

This NPR story looks at the “interview coach” at ESPN and their attempts to bring a high level of quality to their interviewing. Page links to the NPR program and an exemplary interview and critique.

Now, every single editorial employee at ESPN is expected to attend a three-day seminar, where they encounter a lanky, slightly awkward 58-year-old man with little flash. In his efforts to illustrate what he considers the “seven deadly sins of interviewing,” John Sawatsky methodically eviscerates the nation’s most prominent television journalists.

“I want to change the culture of the journalistic interview,” Sawatsky says. “We interview no better now than we did 30 years ago. In some ways, we interview worse.”

There’s a lot to be learned from journalists; not everything applies to other forms of interviewing (say, interviewing users) of course.

I haven’t listened to this yet; checking it out will be first up tomorrow.

[via kottke.org]

Design Research: A Conversation With Steve Portigal (pt. 2)

Here’s part 2 of my conversation with LukeW. Please see Functioning Form for part 1.

Luke Wroblewski

Let me try to clarify by taking a step back. I see field research as a way to remove some of the tarnish that comes with more “traditional” market research like focus groups and surveys. The common perspective is that people in a focus group or survey won’t really tell you what they want or how they do things because often times -they can’t. They are a level removed from the actual activity and as a result may leave out key details or considerations they use the make decisions.

The classic example, and I can’t recall where I first read it, is the washing machine manufacturer that polls thousands of potential customers and asks them “what features do you want in a washing machine?” The responses they get back are: “just the basics”; “i just want a simple setting for colors and whites”; “nothing too fancy”; etc. So the company makes a bunch of no-frills, feature-lite machines and they don’t sell because when it actually comes time to buy a machine the same people that said they want “simple above all else” fall prey to feature-sheen. “Oooh but this one has more features…” I’m sure you’ve heard a similar tale or two.

So what we have here is people saying they do one thing then going out and doing something totally different. Field research should ideally be there at the point of the sale -in context- to enable the company to see what really happens.

Now let’s go back to my original question about digital context. In all the methods you described above -great list by the way!- we’re asking people to tell us what they’re doing rather than being there -in context- when they are doing it.

Maybe I’m picking nits here but I know there are lots of “hidden” subtleties within digital social systems that govern how people behave. There are contexts of when and where that alter behavior. As an example, during a home visit a buyer on eBay may tell you: “I leave positive feedback when I get an item in good condition.” Their actual behavior, however, differs. They may or may not leave feedback based on the type of seller (professional or amateur), how much feedback they have, how much feedback the seller has, the category they are buying from, their intentions for the item after they get it (resell, return), and so on.

I guess when I think of people that spend hours every day immersed in something like World of Warcraft I feel there’s more to their behavior and motivations in that digital space than they can explain in words. How can we be a fly on the wall within that digital context? Or is what I’m looking for already covered by the methods you outlined?

Steve Portigal

Any situation where you have someone telling you about their own behavior is going to include some amount of bias (and let’s pretend for the sake of discussion that our own bias isn’t an additional factor). In focus groups, those influences are hard to leverage (complex peer dynamics, sterile environment, closed-ended discussion), but in contextual research, we can try to take advantage of showing and telling, for instance. Having someone walk through their previous feedback log, and explain, is illustrative of patterns that person may not explicitly be telling us.

Q: Are you leaving feedback for the seller?

A: I leave positive feedback; it’s really important, I usually will look at the condition and decide based on that.

Q: [points to computer screen]Can you walk us through some examples of feedback?

A: Sure, umm, here’s one I left last week. The item was in pretty good condition, so, well, I only left 2 stars, because I didn’t get it right away. And this one here, I left 4 stars because the last time I bought from them it was great. Yeah.

Q: What’s that icon over there?

A: Oh, there’s a bunch of items awaiting feedback.

Q: How often does that happen?

A: Well, I’ve got about 35 in there, some of them let’s see oh yeah, some of them go back about 3 months, I guess.

Now, I’ve totally made that up to support my point so let’s not treat it as data, but as a likely scenario for a dialog in an interview. So much of the process involves triangulation – asking the question at different points in the interview, getting demonstrations as well as declarative statements as well as stories. When you come out of the session, you have to ask yourselves what you think that person’s approach to feedback is. And it lies somewhere in between, but it’s ultimately an interpretive answer.

We can measure people’s TV viewing habits, let’s say, with a Nielsen box, and we can ask them about their behavior. And history shows (like the washing machine example) a disconnect. People under-report their TV watching. It’s easy to think about why; TV is bad, it’s better to show yourself as someone who reads books and goes on hikes than it is as someone who watches a ton of TV. The insight comes out around the delta between the observed and the reported. Of course, not every gulf is an insight, or one that you can use.

And maybe to make it more simple, it’s easier to talk to people about what they did in the past, and why (i.e., leaving feedback) rather than their overall attitude. People make generalized statements but the actual examples contain a lot more subtlety. This of course leads to one reason that companies like to reject any form of customer research – because people can’t talk about what they will do or what they will want. My short answer to that is that the researcher is the one doing the interpretation; in all of this we aren’t simply collecting responses verbatim – we are dynamically choosing different questions and making inferences in order to build our own model of how people will behave.

That said, I had an interesting conversation the other night with Zachary Jean Paradis, a student at the Institute of Design. He described how he had tried to do an ethnography of World of Warcraft (a MMORPG), where presumably a lot more of the behavior to be understood is taking place inside a virtual world. He felt like his traditional tools of ethnographic research didn’t hold up, and he was wishing for another month to better refine his methods. Gaming is an interesting example (and one where I don’t have a great deal of personal experience) of the online-behavior-studied-offline that we’re talking about. I’ve heard that some researchers will videotape the faces and body language of people while they are playing; I imagine you could play those back along with the matching gameplay and have people reflect on what they think was happening at the time. You can see that technique in Gimme Shelter, where Mick Jagger is watching the footage from Altamont while they interview him. A UK company called Everyday Lives does this sort of user research exclusively, preferring to passively observe, and then only interview when there’s a video record of the event to be discussed. I think it’s an interesting tool, but I think we need an ever-expanding palette of methods to deal with new situations as they emerge, rather than dogmatically rely on a single approach.
Am I still dodging your question? Or are we any closer?

Luke Wroblewski

Maybe I’m dodging your answer! One thing you said, though, really resonated with me “getting demonstrations as well as declarative statements as well as stories.” Since we can’t actually be a fly on the wall within complex digital systems yet -and I say that because the tracking software and log analytics software I’ve used is still a ways off from being nuanced and effective enough to match what we can do it the real world- that’s how we need to understand context: through demonstrations, declarations, stories, and of course observation of what people are actually doing on screen. Personally, I do think as digital environments become even more immersive and complex, we’ll need additional methods.

That said, let’s jump into the other topic I wanted to bring up with you. Without getting into pure semantics, why do you think a lot of ethnographic or field research is being characterized as “design research”? Is it user experience design teams within large companies trying to own the research process/data? Is it an attempt to differentiate the type of customer insights a human-centric problem solving approach can uncover from the types of insights Marketing departments have traditionally owned -like customer segmentation? Or does this type of research intrinsically belong in the “design world”?

Steve Portigal

I agree that there’s more to behavior than can be explained in words. It’s up to us to look for the deeper meanings between the words – what is said, what isn’t said, and how it’s said. As far terminology goes, I agree with your suggestion that the label can be an attempt to distinguish the methodology and/or the results from market research, and the departments that do market research. I’m so frustrated by the chaos around methodological labels. I’m sure within organizations they can create a locally-relevant nomenclature (they can, I’m not sure they do) but once you leave the boundaries of their company (through any industry discussion, conference, or online group) they end up sowing confusion. The vendors, of course, who move between companies are even more guilty. There’s a desire to differentiate from the other providers by claiming some proprietary take on doing research: Context-Based Research, PhotoEthnography, Rapid Ethnography, etc. (some of those may be actual methods claimed by actual firms; others may just be me riffing). It’s tough to balance exploring the ideas and staying “on-message” isn’t it? I guess that’s why I don’t take kindly to the terminology wars; they seem to make it more confusing for people.

Luke Wroblewski

So this time you did dodge my question! I’ve consistently heard “design” added to “research” when describing the type of activities we’ve been discussing. Any thoughts on the inclusion of the design label? I know you find yourself in lots of designer-focused events like Design 2.0 in San Francisco, Core77, Overlap… is there really that strong a connection between design and ethnographic research? Why doesn’t this type of research feed business models more than mock-ups? From my experience, the designers eat this kind of data up, the business folks are slow to act on it. What’s you take on that?

Steve Portigal

One suggestion is that the term is historical. Bringing the tools of ethnographic research into product development was led by a few firms that were self-described design firms (like my old company, GVO) or that had ties to design (Doblin, with their connection to the Institute of Design). I would also say (and this is a gross generalization) that market research tends to focus on the evaluative and design research tends to focus on the generative. That’s more about the goals of the research sponsors than anything inherent in the methodologies (since ethnography is ethnography).

And I think those goals or orientations that differ by discipline will affect the gusto with which they eat this stuff up. In many cases, designers are faced with tangible goals. They are committed to acting, since the product has to be designed, and is going to launch. The information they gain from research can help them solve a near-term problem (i.e., what is the organizational framework for a navigation through a space?). Even though my strategic recommendations can be as tangible as my tactical ones, they ask for actions that are much slower (i.e., launch a newsletter that addresses the transparency concerns of customers), more tangled up in organizational (same example) and resource issues (same example), and with many degrees of freedom to create a good solution (and as non-designers, that can be paralyzing).

Luke Wroblewski

Thanks Steve!

Design Research: A Conversation with Steve Portigal

Over at Functioning Form I’m in conversation with LukeW.

To help me work through some recent thoughts I’ve had about Design Research, I asked Steve Portigal -founder of Portigal Consulting and all around bright guy- to talk about context within digital products and the connection between ethnographic research and design. Part one.

Part two will be here (although where “here” is remains in slight flux as this blog is soon to move to portigal.com but is not yet ready. We’ll announce the move when it happens and we’ll make sure you find part two of the conversation!

BW on ethno

BusinessWeek has a new article about ethnography. The author posted a blurb about it on a mailing list I’m on, asking for feedback (I guess some on the list provided input into the piece) and expressing interest continuing the conversation. So far my comments have gone unanswered, so I’m summarizing them here.

It’s nice to see some fresh examples of success in the application of ethnography. The GE example is very cool and goes beyond the usual fix a product case study and into the evolve a business’s culture that really rang true from my own experience.

However, I was disappointed to see the article buy into the ethnography = anthropology myth and the corollary that all ethnographers are anthropologists. Indeed, the article incorrectly attributes the anthropology credential to some people such as Tony Salvador who I believe was trained as a psychologist, or the people at Steelcase, some of whom I know as graduates of the Institute of Design, and are definitely not anthropologists. IDEO may have anthropologists, but a great deal of their people involved in “human factors” (as they term it) are coming with other educational backgrounds.

It’s tempting to see a conspiracy of highly-placed anthropologists who work behind the scenes to ensure that any conversation about user research in product development and consulting succumbs helplessly to this myth, but I think really sloppy reporting is more likely the culprit here.

John Thackera Thackara writes about the article in his typical sanctimonious style (seriously – I will have to give up on In The Bubble because it’s filled with mean-spirited judgment of one profession or endeavor on one page, and then a capricious about-face on the next page to drool over another effort that meets his opaque standards).

Do ethnographers need exotic names to do well in business? A story in Business Week features two guys called ‘J. Wilton L. Agatstein Jr’ (who runs Intel’s new emerging-markets unit) and ‘Timothy deWaal Malefyt’ (an anthropologist who runs ‘cultural discovery’ at ad firm BBDO Worldwide).

Whoah. Racist much, John? Portigal is a pretty funny name. So is Thackera Thackara. What of it?

Ethnography and new product development

From Innovation Weblog (via The Business Innovation Insider)

Simply put, ethnography – as it applies to innovation – is the process of doing observational research, going into the field to watch how customers utilize your products. Often used in consumer new product research, ethnography is an excellent way to uncover new opportunities for product improvement.

For example, speaker Pam Rogers, who is corporate director of global customer excellence and innovation, explained how the inspiration for a pedestal/storage unit for its Duet front-loading washers and dryers came from observing a woman who had placed her Whirlpool dryer upon cinderblocks, to make it easier to load and unload it without having to bend over.

Okay, yes, I guess, but really, no. It’s not simply about observation. That seems to be the easy part to explain and so that’s the part that gets spoken about. I’ve written a bit about ethnography here

So often, companies go to the trouble of studying customers, only to address the opportunities revealed by usage. For example, an award-winning snow shovel was redesigned when the design team went out to watch how their product was being used, found that women instead of men were shoveling, and so they made the handle smaller.

But there’s much more that can be revealed. What is the shoveling occasion (or, if you will, ritual) really about? What meanings does it hold? Does it hearken back to childhood? Or does it represent female independence? Or the nurturing of motherhood? Or the abandonment by men? Probably it’s none of those, but the point is that within the ordinary activity of shoveling we can find deep meanings that can provide enormous opportunities for innovation as we question the basic assumptions about what the product could possibly be.

I’ve found the word ethnography to be a troubling one, frankly. It’s a mouthful, it reeks of academic snootery and hand-waving inconclusiveness. It’s gets confused with anthropology and various parties have tried to claim the pure methodology only for those with the right doctorate. And I’ve been an advocate for stepping aside from the word and pointing to the key elements (getting out of your own context, observing and interviewing, and synthesizing something new). But that is troubling for some.

Grant McCracken has written a strongly-worded piece about the coming-of-age of ethnography in business in 2006, and there’s a spirited discussion in the comments below his piece, including several entries from me, including one where I advocate ignoring the word and just getting to the root of it (as I said above). Grant doesn’t take too well to that.

It’s a very troubling issue that is perhaps eating away at the development of an excellent practice and community of practice around that excellence. But I do think the terminology wars and the discipline battles are painful, frustrating, and perhaps fruitless. I look at the “interface” community which has split into many different professional networks based on what term they agree with (IxD, UxD, UX, UD, IX, ID, etc.) or what end of the egg they prefer to break open.

Yesterday I was in a conference call with a prospective client. We were proposing some work and hadn’t used the word ethnography at all. An internal person from another part of the organization was very interested in displaying her own mastery of the research process, and made numerous references to some ongoing work as “my ethnography.” Only she couldn’t even comfortably pronounce ethnography. And she wasn’t doing it; she was sending it out to the “only” provider that did this, apparently (?). And what were they doing? Inviting cool kids to an art gallery in Miami. [Okay, I don’t get this at all].

At a conference the other week I participated in a side conversation that included this snippet “Oh that’s not ethnography, that’s just depth-interviewing.”

I may be coming around to Grant’s way of looking at this. We have a problem. I’ve got my explanation, sure, but so does everyone else, whether they have more experience than I do, or worse pronunciation than I do. We’ve got experts like the Innovation Weblog getting it badly wrong, Pam Rogers perhaps missing some of the point, my recent encounters (presumed experts in their own peer group?) with their own versions of what we’re doing, and on and on.

Unfortunately, I have no solutions. And I don’t see a culture that is ready to reach a solution, establish a common language, speak in one voice (not millions), establish standards, or even work together on this.

Interviewing and research tips

Dina has a bunch of great tips for people wanting to be better at user research/interviewing.

# reading between the lines — dont just go with what they ‘say’ – look for non-verbal cues that really tell you what they ‘feel’. Also, try and understand the rationale behind what they say – laddering down to end values is something that always helps. It doesn’t pay just to know that a Toyota Corolla = Amitabh Bachchan – we need to understand why the analogy is made
# agility – you’ve got to be so quick in your mind – pick up cues from what respondents say – and take them forward. Listen well and react quickly – you should never feel when you listen too your tapes – oh how I wish I had probed this a little more.

Primping for the Cameras in the Name of Research

Cosmetics companies are striving to understand how their products are used differently in their various emerging markets. Presumably, they are elsewhere looking at differences in meaning, in addition to simply understanding usage.

Crucial to that effort is the search for differences that could help build a brand in critical emerging markets like India and China. L’Oreal has an expanding network of 13 evaluation centers around the world created to observe grooming and ponder a variety of burning questions: Do national differences exist in primping styles? Would women in Japan and Europe, for instance, stroke on mascara with the same lavish hand? (The answer is that in Japan, women apply mascara with an average of 100 brush strokes compared with Europeans, who are satisfied with 50, a difference noted by ethnologists for L’Oreal.)

It was observations like these that ultimately affected how the company made and marketed its mascaras or developed the foaming quality of its shampoos. “We are far from understanding everybody everywhere. It takes time,” said Fabrice Aghassian, director of international product evaluation for L’Oreal, which is seeking to map the world’s beauty routines in a landscape the company calls geocosmetics. “When we know the behaviors of people, we know what unexpressed expectations we do have to consider.”

Little lies by focus groupies are costly

Nice article about people that lie in order to qualify for market research studies

Researchers call these truth-stretchers focus groupies, a sneaky cadre that adopt multiple identities in order to secure paid seats on the dozens of focus groups that meet every week in the Bay Area.

Firms pay $50 to $100 cash for an hour or two of work that usually involves a moderated discussion about a new product or service with up to a dozen people gathered in a room equipped with a two-way mirror.

The allure of easy money leads hundreds of people every year to treat focus groups as a source of nearly work-free income. Get-rich-quick schemers even advertise focus groups as a source of cash.

And if it means telling a few lies along the way about your favorite brand of frozen pizza or the number of times you have already participated in a focus group, well, it’s no crime to fib to a marketing company.

Researchers go to great lengths to weed out groupies, including the use of exhaustive database cross-checks to ferret out the ‘cheaters’ and ‘repeaters,’ along with detailed screening interviews. Competing firms even share groupies’ names in the reverse form of a ‘do not call’ list.

‘It’s bad for the whole industry so we cooperate with each other,’ said Nichols Research Group Vice President Jane Rosen, whose Bay Area firm purges several hundred groupies a year from its database.

How far will people go?

They sign up with aliases, usually derivatives of their real names with different initials and middle names, Rosen said.

They may use a post office box address under one application and then a home address for the second response.

‘We had a woman sign up for two focus groups on the same day and after she finished the first session, she went out to her car and changed into a new set of clothes and put on a wig,’ Rosen said. ‘Fortunately, one of our people thought something looked wrong about her.’

Q&A Research in Walnut Creek recently foiled a woman who claimed to own a particular brand of luxury car, but the name on the automobile registration she provided did not match her own.

‘We had another man who used his first name for one group, then his middle name for a second group the next day and then a third one the following week,’ said Eric Tavizon, Q&A’s focus group project manager. ‘One of the clients caught him because he mistakenly signed up for events by the same sponsor and they recognized him.’

Of course I’ve encountered this on a much smaller scale; so much of what I do is predicated on a basic foundation of trust (and trust goes in two directions, of course) and it’s lurid and disturbing to consider how that trust can be violated (when do we read the piece about the rapist who posed as an ethnographer to gain access? yikes).

I’ve started a discussion thread on Discovery about this; we’ll see if anything develops.

user centered design – sort of – in NYC subways

full story

Mr. Malave was one of dozens of curious riders who attended an ‘open house’ sponsored yesterday afternoon by New York City Transit to show off and receive feedback on a five-car test train, a prototype of the R160, the newest generation of subway cars.

Riders yesterday, told to focus on the FIND panel, were asked questions like, ‘Do you feel reassured that the train is going to your station?’ and ‘How easy or hard is it to read the words and letters on the sign?’

But riders seemed to be paying less attention to the sign than the rest of the car. Some of them said they did not regularly take the Nos. 2, 4, 5 and 6 lines (which use R142 cars, similar in design to the R143) or the L line and so were not familiar with the latest design.

Asked to compare the new car with the F train that she normally rides, Mar?a Romero, 72, a retired nurse’s aide from Gravesend, Brooklyn, said, ‘This is three times more advanced!’ Jared M. Skolnick, 34, an Internet marketer from the Upper West Side, said he admired the bright fluorescent lights, since he often took photographs in the subway.

James V. Sears, the agency’s senior director of marketing research, said the results of the surveys – along with comments from focus groups convened in 2003 – could be incorporated into the final design of the FIND panel.

Right. Because in order to understand the reassurance of a design feature, you simply ask people if they find a certain feature to be reassuring? Sigh!

Love your test participants more than yourself

Wonderfully passionate blog entry about making that all-important connection with another person in a user-research setting. This would be great fodder for the workshop I’m leading at EPIC next month.

Last week, after a long long time I had a chance to conduct user interviews again. I loved any minute of it. There is nothing more rewarding (for me) than spending two hours with people I never met before (and probably I will never meet again) trying to understand the world from their point of view.

In those two hours and from the first few seconds, my attention is totally focused on the other person. I observe how they enter the room, how they look at me, and how they shake my hand; I need to understand anything I can about their personality, their level of comfort, and their communication style to be able to be in synch with them. The entire session is a dance, where I ask and listen, probe and observe, with the only purpose of gaining insight in somebody else perceptions, thoughts, and expectations. It’s always a fascinating journey.

….

But I believe that the magic of understanding another person is not just a technical issue. It requires to suspend for a moment our ego-centered way to interpret the world and open up to a different interpretation. In a way, it’s about love.

There is something wonderful in experiencing somebody else’s world. You understanding expands, you suddenly see something you could not see before. And there is no going back.

Rambling thoughts on “User Research Strategies: What Works, What Does Not Work”

Last night’s BayCHI event was a good experience. A panel of champions of user research (Director, Manager, Lead, etc.) at key Silicon Valley companies (Google, Yahoo, Adobe, Intuit, and eBay) attracted a large and energetic crowd. The pre-meeting dinner was extremely well-attended. It felt like the discipline is having a good moment in the zeitgeist.

Each panelist gave a 10-minute summary of what’s going on at their firm. What types of methods they are using, how they are feeding design, strategy. How they might interface with market research and other areas of the business. Where they have been, historically, and what may have changed in how they are embraced (or not).

Rashmi Sinha moderated, and only spent a bit of time asking her own followup questions after the panelists finished; and then it was basically an hour (?) of audience questions. Although I stood up and asked a question, I would rather have had more questions from her and less from the audience (that’s my bias, I guess, as someone who’s played that role in the past); the audience questions are not usually about creating conversation, and the moderator is obviously in a role to do that. By the time we finished with questions people were asking about how to recruit participants for studies; a tactical question that had no place in this meeting, if we were there to discuss the practice and how it integrates into corporate America, then let’s not deal with newbie process questions. I’m not minimizing the importance of that question to the person who asked it, but it wasn’t on topic and kinda brought things down for me.

People mumbled afterwards about wanting to see some conflict between the panelists, who represented competitive firms (but maybe didn’t see themselves individually as competitors) and who sometimes expressed different points of view on how to use the tools of user research (it’s hard for me to be specific from memory, but there were several examples from Google about how user research wasn’t always necessary, but they were ridiculous examples, as in, “should we have told people not to build a search interface until they had done years of research” as the strawman questions, when I don’t think anyone was advocating years of research, more so the opposite). It wasn’t in panelists charge to debate what they heard from the others; they were there to tell their own story and they all did that very well.

Perhaps the comments about conflict are proxies for my desire for more conversation; something that (as user reseachers know) takes good questions, and frankly, audience members just aren’t going to ask good questions. This sounds terribly snobby and let me clarify – there are questions that are informational (what type of deliverables do you use? how do you recruit) and there are questions that provoke conversation and interaction.

There’s another panel phenomenon at work here – “question drift” – whoever answers the question first is the most on target; as other answers come from the panelists, we end up hearing about an entirely different question, and we’ve lost the thread. I don’t have a solution to this. Sometimes the drift is interesting, but often it’s just a bit frustrating.

So it was hard to take much specific away from the evening – there was a lot of info; a lot of bits of perspective and insight and jargon thrown out quickly, with something new on the heels, so I felt like it was an immersion more than an education.

But here’s what I took away:

  • this is a mature field; you can see the newer practices (Google) presented as adolescent next to their more wizened counterparts
  • I’ve lived as a consultant for a very long time; there’s a whole set of challenges and benefits that these corporate folks have that are almost alien to me – I felt very aware of how I can deal with some of their situations so much more easily, and how there’s so much more formalized and permanent processes being created that I’m not at all engaged with
  • it’s just tradeoffs and contrast, one isn’t better than the other, we need to be in-house and out-of-house both
  • it’s not clear to me how research is different from design (and I don’t mean Research and Design) – this was my question and I don’t know that I got an answer except a fallback to corporate structures (and one person pointed out that designers and researchers have very different skill sets, but that wasn’t my question – if this is collaborative work, can’t teams of people with complementary skills deliver ONE thing – “design” – rather than breaking it down so much) and formalized processes
  • this is a bit of a hot topic among software/tech/design types right now
  • MORE I REMEMBERED: Christian Rohrer from eBay defined success criteria for user research as impact (which I really liked), and he defined impact as
    1. credibility
    2. consumability
    3. relevance

BBC Radio show on ethnography in business

Shop Talk is a BBC radio show featuring a discussion of ethnographic research in business. Thanks to Michael Andrews and Louise Ferguson for this.

It was a bit challenging for me to listen to; Genevieve Bell was the only person who (in a very radio-unfriendly fashion) tried to move the sound-bites into a discussion, usually to be squelched by the host. There was just too much to cover in 28 minutes – too many people giving their spiel about what this is, why it’s necessary, who is qualified (or not) to engage in it, and in a couple of cases some examples of what was learned and what the implications were. Definitions of terms such as “ethnography” and “anthropology” were not common amongst the panelists and created what I would imagine was some confusion among non-industry (ie, the user research industry) listeners. Me, I was just annoyed by that discussion.

Anyway, there are some nice bits in there, and it’s pretty good to hear some different voices on the topic, and for me, to hear my profession discussed from a UK business perspective, that was quite refreshing.

What People Want In Their Homes and Communities

The NYT writes a front-page story about the growth in housing developments in the US in areas that were formerly “the middle of nowhere.” Beyond being generally interesting as a trend, I was intrigued by the (perhaps not novel but at least unique to me) teaser of how they are figuring out what to put into these homes.

One area in which KB Home takes pride is its market research. It asks things like where people want their kitchens and how much more of a commute they can stomach. And it surveys its own buyers to get a comprehensive idea of who they are and why they bought.

In its most recent survey of Tampa home buyers, KB asked people what they valued the most in their home and community. They wanted more space and a greater sense of security. Safety always ranks second, even in communities where there is virtually no crime.

Asked what they wanted in a home, 88 percent said a home security system, 93 percent said they preferred neighborhoods with “more streetlights” and 96 percent insisted on deadbolt locks or security doors.

So KB Home offers them all. “It’s up to us to figure out what people really want and to translate that into architecture,” said Erik Kough, KB’s vice president for architecture. And the company designs its communities with winding streets with sidewalks and cul-de-sacs to keep traffic slow, to give a sense of containment and to give an appearance distinctly unlike the urban grid that the young, middle-class families instinctively associate with crime. “I definitely feel safe here. I feel protected,” said Lisa Crawford, who moved to New River about a year ago with her husband, Steve, and their two children.

“And I can tell you that the people in Tampa are a whole lot different than the people here,” Ms. Crawford said. “In Tampa, there’s a faster pace. I like it here, that it’s more of a community, more of a small-town feel.”

Secret Grooming Habits

NPR’s Marketplace does a feature on ethnography. It’s the new reality TV, I guess. A market research trend, they say. It’s kind of a snide piece, isn’t it weird that people use products this way and they tell us? Eyeew!

I found this page about the main ethnographer they profile

david grzelak
Columbus OH: Advertising agency Ten United recruits David Grzelak for the unusual position of director of ethnography. Previously marketing research manager at Hallmark and a consultant, Grzelak heads a newly formed Behavioral Insights Group that works with clients to gain a deeper understanding of their respective target markets. His degrees are in anthropology and business anthropology from Wayne State and Purdue, respectively. Ten United has roughly $185 million in billings for clients like Anthem Blue Cross and Blue Shield, Limited Brands, OhioHealth, Perkins Restaurants, Time-Warners Communications, Hoover and Prestige Brands including Murine, Comet and Prell.

As usual (this is one of my frequent comments) the press will focus on either advertising or product development, but won’t acknowledge both. This one is about advertising.

Real Women, Real Beauty, Fake Ethnography

pod_photo_millionalbum

Dove has been making a big splash with its recent advertising campaign based on showing Real Women with all their flaws (i.e., a range of body tapes and ages that aren’t typical hair/skin care models), so it’s not surprising that a recent ad for Dove used the aesthetic of ethnographic interviews. This has been done a zillion times, especially in the last few years as ethnography becomes a more common touch point in our culture (and as the producer and the consumer collapse further). I’ve written about this many times, but I’m still struck whenever I see an ad doing this.

The Dove ad involved women being interviewed while they were bathing, and it cut between lower-quality video clips of several different women, with half of the clips being about the product, and half being about the process of being interviewed: “Oh, I’m in the tub, isn’t this a bit awkward?!”; “You’re all up in my armpit now.” were two examples.

I know a fair amount of research does get done in seemingly impossible settings such as the bathroom, but I’ve never been directly involved in such a study myself. I did see a Whirlpool presentation many years ago about how they did such a study (i.e., people wear bathing suits) but overall it sounds pretty fun just for the added challenges of establishing a comfortable rapport in such a socially awkward setting.

Anonymous Responses Are Useless

From Garrick Van Buren’s Work Better Weblog

One of my current projects has a major survey component. The survey ends with:

If you’d be open to follow up questions, enter your email address below.

There’s about a 60 / 40 split on responses with emails and those without. The responses without email addresses have skipped questions, irrelevant answers, and are generally unusable. This is so much the case, that I’ve found it a better use of time to check for an email address first – then read the response.

It’s interesting that people comfortable with being contacted give useful answers, while those providing non-useful responses don’t provide a way to be contacted.

Conventional wisdom on requiring accountability has it backward. Accountable people want to be responsible for their actions. Those that aren’t don’t. Forcing it doesn’t change anything.

This is purely anecdotal of course, and may not generalize to other surveys about other topics, but nonetheless seemed an interesting data point.

A mobile tale of three cities

The International Herald Tribune reports on a cultural study of mobile phone use in Europe.

“Europe is looked at as a broadly similar market,” Lasén said. “But in studying mobile phones you can see details in each country can change enormously.”

For her research, Lasén combined individual interviews with street-level observations in each city in both 2002 and 2004. Interview questions ranged from mobile phone habits to people’s relationship with the device. Her observation centers were a major train station, a commercial area and a business district in each city.

I thought everyone did a study like this in 1999 – 2000, but I guess they are continuing (not that there’s not more to be learned; just that these sort of things are sometimes fashionable, if you will). I was involved in a fascinating comparison of French and Japanese mobile phone usage and attitude.

Via Future Now

Series

About Steve