Posts tagged “war stories”

Wisdom from “The Americans”

On the season finale of The Americans, there’s a bit of a post-mortem (so to speak) on a mission, including this bit of dialog

The people back home who aren’t in the field, sometimes they get what we do and sometimes they don’t. But when you’re in the field you have to make split-second decisions. You don’t always have the luxury of thinking things through every time…it’s important to be honest about mistakes. But acknowledging them doesn’t always keep them from happening again.

Obviously they are referencing a different sort of fieldwork. But the lesson applies, nonetheless. For more on this theme, check out my new book Doorbells, Danger, and Dead Batteries: User Research War Stories.

Steve’s War Story: Details Disconnect

This story was originally published on behalf of The Field Study Handbook.

Last year I was working on a project for a financial technology client. Finding participants is often a challenge, but on this project, for small business owners, it was particularly difficult.

We had hoped to base this research on previous studies, but it proved difficult to glean details about how previous studies were done. There were rumors that another team, elsewhere in the country, had developed a segmentation algorithm, but voicemails and emails went unanswered. We heard about great participants from previous studies that we should revisit, but no one would get back to us. The schedule ticked by and the pressure mounted. In the end, we were left with no choice to work around these limitations. Finally, I began to approach recruiting agencies.

My go-to recruiting team refused to take the assignment on as they had, ironically, recruited for one of these previous studies and felt like they had tapped out the local market. Another company had gone out of business, and a third didn’t think they could accomplish the recruit.

I ended up with a recruiter I had never worked with. In the end, I think they did a good job, but a new relationship added stress to the increasingly complex recruiting process.

In our introductory call, one of our recruits expressed surprise and concern that there would two of us visiting his very small office. We eventually agreed that even though it might be cramped, it would be okay. The recruiting agency, when asked about this disconnect, reassured me that they made it clear, as per my instructions, that there would be two of us. I was confused, as the participant had insisted they had never told him anything about this.

Later that day I got an email from the participant, who sought reassurance about the purpose of the interview. He had clicked on my website (seen in my email signature) and was concerned that I was actually going to be pitching him my services. He had been involved in a focus group through this agency before, and presumed this would be something similar. I confirmed that this was not a sales pitch.

A few days later we met with him in his exceptionally cramped one-person workspace. As the interview unfolded, he abruptly stopped and directly, yet politely expressed confusion and discomfort about the interview itself. Why were we asking these questions? Who do we represent? How are we going to use this information?

It took a long, unhurried conversation about the process and our objectives to put him at ease. We resumed the interview and learned a great deal about his truly amazing businesses, past, present, and future.

I emphasize his politeness in stopping the interview, because now, when I go back to the transcript, that’s what I see. But at that time, sitting in that interview, it didn’t feel that way. It felt aggressive and angry and I spent the remainder of the interview feeling uncertain about our rapport. I overcompensated with excessive deference, people-pleasing, and probably flattery. That’s not a comfortable feeling and it’s not conducive to a good interview. I have empathy for someone feeling uncomfortable about something as odd as two strangers with a video camera coming into their office space to ask about their professional history. It’s easy to mischaracterize people that don’t “get it” as difficult. And I assume that I am pretty good at managing expectations at all the common points of failure in establishing rapport.

But boy it’d be nice if we had someone to blame. That guy was a jerkface! The recruiter didn’t do their job (and then lied and insisted they did!). Steve didn’t handle the first call or the interview kickoff properly! Yet it doesn’t seem like any of these are true.

While I felt sheepish at the end of the interview, I was surprised to get a LinkedIn request from the participant immediately afterwards. And, I guess, less surprised when I heard from him a few times weeks later about not receiving his incentive payment (This was one of the very few studies where I asked the agency to send checks after the interview was completed, rather than handing people the incentive directly myself. Mistake? I don’t know). When I followed up with the recruiter about the missing incentive, I heard in some detail how this participant had already called and yelled at the admin staff.

And so it goes.

DDD: Link Roundup


It’s been two months since Doorbells, Danger, and Dead Batteries was released. Here’s links to presentations, podcasts and more stuff related to the book.

The Book

Reviews

Podcasts

Interviews

Presentations

Other

Noël’s War Story: Truck Stop

Noël Bankston is a UX Research Lead and Human Factors Engineer at Zebra Technologies, currently living in Queens, NY. She told this story live at the Interaction 17 conference.

“So Jim, what would you like to do for lunch? “My treat!” It was the moment I had been dreading all day, ironic since I am a lover of food. I was trying to sound chipper but I was worn through.

It was 2 pm and I was starving. I was sitting in the cab of a 48’ tractor trailer in Lowell, Arkansas. This was my first “ride along” research trip and I had not come prepared with snacks. I was doing in-depth generative research of the pick-up and delivery process for a freight company and hadn’t known that we don’t have lunch until all the deliveries were completed.

I was also not prepared for the weather as I am from up north and I thought the South would be hot in late May. It wasn’t – it was a constant drizzle and cold. So I was sitting in the cab feeling small and tired in the oversized loaner jacket that the dispatcher had given me. We had been on the road since 8:45 am but I had arrived at the trailer dispatch site even earlier to observe the set-up process. And that should have been fine, because on a normal day, Jim finishes around noon. But today we saw all the exceptions – an unprepared customer, incorrect paperwork, an obstructed delivery dock, and poor routing. As a researcher, it was a gold-mine as I observed where problems occurred and how Jim handled them. But as someone who is mildly hypoglycemic, it meant I was getting hangry. It had been a long morning of climbing into and out of that cab, learning which hand to place where to get the right leverage to pull yourself up as you step onto the step that is only wide enough for half your foot. And I don’t know how many of you have ridden inside of a tractor trailer but it is loud and you feel every bump.

In that moment as I asked about lunch, damp, tired, and hungry, I thought back on the the anxiety I had felt earlier in the day about lunch. A co-worker told me that on his previous ride-along they had eaten a burger from a gas station mini-mart. Even on a normal day that would make me uneasy, as gas stations aren’t known for freshness and hygiene. I knew that this type of research means being available for wherever the subject takes you, but I was really hoping that didn’t include food poisoning.

But at this point, 8 hours from my previous meal and having no idea what part of town we were in, who was I to be picky?

“So Jim, what would you like to do for lunch?”

“I just want a salad. I try to eat healthy.” I gave a huge sigh of relief, accompanied by a rumble of rejoicing from my stomach. It seemed that between the two of us, I would be eating the bigger meal. I found a nearby Mexican restaurant on Yelp. While enjoying the flavor combination of fresh cilantro and lime with nary a fryolator in sight, I realized how I had been making assumptions about “truckers” based on stereotypes rather than letting the research reveal the truth. And those assumptions were also judgments about health and lifestyle. Jim was aware of the health effects of his job and wasn’t going to turn down an opportunity to have a healthful meal, especially when a researcher was paying! One of the reasons truckers eat unhealthy food is both cost and convenience. Truck stops get food fast and are less expensive. Unfortunately, our food system is set up in a way that fresh, whole food costs much more than highly processed, industrially produced food.

I won’t be able to eliminate all my biases or preconceived notions but I can grow in my awareness of them. I have been on many more ride-alongs and other types of research trips since then. You better believe I always have a granola bar with me.

Elizabeth’s War Story: Ramping Up

Elizabeth Allen is a UX Researcher at Shopify, an ecommerce platform based in Canada. She told this story live at the Interaction 17 conference.

A few years ago, I was working at Centralis, a UX research and design consulting firm in the Chicago area. One of our clients was a public transportation agency, and our project involved testing the maps and signage within and between transit stations by accompanying participants as they completed realistic wayfinding scenarios to try to get from station to station and find their correct train or bus.

As part of this testing, my research partner Kathi Kaiser and I included individuals with motor and visual disabilities to make sure they were able to navigate just as well as those who didn’t have these challenges. One participant, Susan, was in a motorized wheelchair, and we began our session with a scenario that had us traveling to a station and accessing an elevated platform where she would wait for a train.

Chicago summers can be very hot and humid, and this was one of the hottest of the year. We were all sweating by the time we got to the station even though it was just a short walk from the coffee shop where we met to start the session. Now, this station had no elevator; instead, outside the station was a very long ramp to reach the platform. This was probably the longest ramp I’d ever seen at a transit station — it had two or three switchbacks just to reach the top!

We started up the ramp, and when we were about halfway up, Susan’s wheelchair started slowing down. “Uh oh”, she said. “I think my battery is about to die. I totally forgot to charge it before I went out, and steep ramps like this always make it run out faster.” Sure enough, a few seconds later, the wheelchair slowed to a halt, completely dead.

At this point, we had to make a decision based on what was best for Susan and for the research: do we end the session early, push Susan’s chair back to our starting point, and explain to our client that we would miss out on gathering valuable accessibility insights, or do we see if we can find a power source and salvage what we can of the session? We explained to Susan that we could either end the session or try to keep going, and luckily, she was still excited about the session and was game to push on — literally.

After wheezing our way up the rest of the ramp, dripping with sweat, we got to the platform and found no electrical outlets in sight. The ticket counter was also closed, but after a lot of roaming around we were able to find the lone janitor. We were very fortunate, because he was extremely kind, and offered to let us plug Susan’s chair into an outlet in one of the back rooms.

This story ends happily. After a half hour or so, Susan’s chair was charged up, and during that time we were able to improvise some interview questions and short scenarios we could talk through with her while we waited. It really helped that we were able to think on our feet and that we had a participant who had a positive attitude and was interested in the session. Overall, we were able to salvage a research session that was difficult to recruit for, and our client was really happy with what we learned.

Five Questions with Steve Portigal

Here’s “Five Questions” with me about war stories, user research, and the O’Reilly Design Conference. Click for the whole thing; Below is a short excerpt.

Why is user research so hard to do well?

I talked about this a bit in Interviewing Users—that the assumption we can just use our social defaults because it’s just “talking to people” holds us back from being better at user research. We have to unlearn a lot of patterns (e.g., sharing about yourself) in order to get to a very different outcome (a good session versus a good hangout). In looking at war stories, I’m digging further into the challenges we face in doing research, and hopefully not stating the obvious but research with other people will have massive elements of unpredictability in it. That means we learn what we didn’t know we didn’t know (and would never otherwise have thought to ask about), but it also means that our attempts to plan and control the process are somewhat foolish (and yet, someone who does research without planning is obviously a fool). There’s an element here of temperament, or worldview, that isn’t so natural for everyone. In some of the stories I’ve gathered, people do everything right, and yet things still go wrong. That’s not a welcome truth.

Learning from the comic, tragic & astonishing moments in user research (transcript)


Last week I did an online chat with UX Mastery about Doorbells, Danger, and Dead Batteries. See one snippet below, but please check out the entire transcript here.

crystal: Do you think that approaching the interview with some small vulnerability of your own allows them to be more vulnerable as well and open up and give more insight? And have you found that added insight to often add value to the research?

steveportigal: our own vulnerability – that’s fascinating and I don’t have a clear take on that. I think a shallow reading says being vulnerable means sharing about ourselves and I am mostly against doing that most of the time for most researchers but it makes me ponder what’s a richer more nuanced sense of what our own vulnerability is, if by being still, present, focused, listening, and not needing to make it about us, we might convey some vulnerability. I think it’s meeting people where they are, accepting them where they are and not putting ourself into it. Which – to your point – feels DAMN risky to a lot of people. Set aside your agenda and listen but do so in a productive effective you’re-on-the-job way, so you are balancing different forces and risks.

I dunno, is that ‘vulnerable?’

Stories from the field: An interview with Steve Portigal

Gerry Gaffney interviewed me about Doorbells, Danger, and Dead Batteries (which he contributed to) – and user research in general – for his User Experience podcast.

Check out the podcast on iTunes. Or listen via Gerry’s site (which also features a transcript). Or, listen below.

The idea that some of these ideas are metaphors for life I think is absolutely true and, again, I can sound kind of highfalutin and pretentious here but I think the thinking that I went through in this book is looking at… some of these external factors, right? You know, make sure your camera is ready and you don’t break the cable and you know the sort of “equipmenty” type things that we have to think about. But so many of these are about what do we do when the unexpected happens and acknowledge the unexpected is going to happen and that those are definitely life skills. And I think one of the takeaways that I come back to several times, and I just alluded to it a minute ago, which is know when to walk away. You know and so when you’re in a situation do you keep trying to turn that situation from a failure into a success or do you say “You know what? This isn’t going to work,” and you leave.

What’s New: Doorbells, Danger, and Dead Batteries

doorbells-danger-dead-batteries-cover
My new book comes out today and Lou Rosenfeld has an enthusiastic appreciation and a bit of the back story about the book’s journey.
Read it all here

I dipped into about a dozen of the 60+ field research war stories that make up the bulk of the book. The stories do what stories are supposed to do: engage. And the contributors have been through some experiences that will make you laugh, sweat with fear and discomfort, and—let’s face it—enjoy a bit of schadenfreude. But it’s wrong to see Steve’s new book simply as a compilation of user research war stories. In Doorbells, Danger, and Dead Batteries, Steve comes through: he delivers a broader framework that’s useful for making sense of user research—or, actually, situations with people.

Doorbells, Danger, and Dead Batteries available for pre-order

doorbells-danger-dead-batteries-cover
My new book comes out on December 6!

You can pre-order (with a discount) from Rosenfeld Media (or from Amazon). Also coming soon is an audiobook version! Remember, your review on Amazon really helps drive awareness.

————————————-
Doorbells, Danger, and Dead Batteries
User Research War Stories

User research war stories are personal accounts of the challenges researchers encounter out in the field, where mishaps are inevitable yet incredibly instructive. Doorbells, Danger, and Dead Batteries is a diverse compilation of war stories that range from comically bizarre to astonishingly tragic, tied together with valuable lessons from expert user researcher Steve Portigal.

The stories Steve Portigal knits together here have an extraordinary and immediate intimacy, like listening in on 66 researchers’ bedtime prayers. Anne Lamott says there are essentially three kinds of prayers: help, thanks, and wow! Doorbells, Danger, and Dead Batteries covers the whole range, with humor and wisdom.

Dan Klyn, information architect, co-founder of The Understanding Group (TUG)

See more testimonials

Support the War Stories for SXSW and more!

Watch this space for a big announcement about the War Stories coming up after Labor Day. In the meantime, I’ve applied to speak about War Stories in design research at SXSW. You can help here (whether or not you are planning to attend SXSW) by creating an account, voting thumbs up for the proposed talk and even adding a comment.

We’ve got three recent stories, all from the Kitchener-Waterloo area: Jennifer’s War Story: Keeping the Lights on in Vegas, Julia’s War Story: For Want Of A Shoe and Susie’s War Story: A Sigh Is Just A Sigh. And coming up next month at Fluxible in Kitchener-Waterloo, I’ll be presenting Epic FAIL: Takeaways from the War Stories Project.

Series

About Steve