This story was originally published on behalf of The Field Study Handbook.

Last year I was working on a project for a financial technology client. Finding participants is often a challenge, but on this project, for small business owners, it was particularly difficult.

We had hoped to base this research on previous studies, but it proved difficult to glean details about how previous studies were done. There were rumors that another team, elsewhere in the country, had developed a segmentation algorithm, but voicemails and emails went unanswered. We heard about great participants from previous studies that we should revisit, but no one would get back to us. The schedule ticked by and the pressure mounted. In the end, we were left with no choice to work around these limitations. Finally, I began to approach recruiting agencies.

My go-to recruiting team refused to take the assignment on as they had, ironically, recruited for one of these previous studies and felt like they had tapped out the local market. Another company had gone out of business, and a third didn’t think they could accomplish the recruit.

I ended up with a recruiter I had never worked with. In the end, I think they did a good job, but a new relationship added stress to the increasingly complex recruiting process.

In our introductory call, one of our recruits expressed surprise and concern that there would two of us visiting his very small office. We eventually agreed that even though it might be cramped, it would be okay. The recruiting agency, when asked about this disconnect, reassured me that they made it clear, as per my instructions, that there would be two of us. I was confused, as the participant had insisted they had never told him anything about this.

Later that day I got an email from the participant, who sought reassurance about the purpose of the interview. He had clicked on my website (seen in my email signature) and was concerned that I was actually going to be pitching him my services. He had been involved in a focus group through this agency before, and presumed this would be something similar. I confirmed that this was not a sales pitch.

A few days later we met with him in his exceptionally cramped one-person workspace. As the interview unfolded, he abruptly stopped and directly, yet politely expressed confusion and discomfort about the interview itself. Why were we asking these questions? Who do we represent? How are we going to use this information?

It took a long, unhurried conversation about the process and our objectives to put him at ease. We resumed the interview and learned a great deal about his truly amazing businesses, past, present, and future.

—

I emphasize his politeness in stopping the interview, because now, when I go back to the transcript, that’s what I see. But at that time, sitting in that interview, it didn’t feel that way. It felt aggressive and angry and I spent the remainder of the interview feeling uncertain about our rapport. I overcompensated with excessive deference, people-pleasing, and probably flattery. That’s not a comfortable feeling and it’s not conducive to a good interview. I have empathy for someone feeling uncomfortable about something as odd as two strangers with a video camera coming into their office space to ask about their professional history. It’s easy to mischaracterize people that don’t “get it” as difficult. And I assume that I am pretty good at managing expectations at all the common points of failure in establishing rapport.

But boy it’d be nice if we had someone to blame. That guy was a jerkface! The recruiter didn’t do their job (and then lied and insisted they did!). Steve didn’t handle the first call or the interview kickoff properly! Yet it doesn’t seem like any of these are true.

While I felt sheepish at the end of the interview, I was surprised to get a LinkedIn request from the participant immediately afterwards. And, I guess, less surprised when I heard from him a few times weeks later about not receiving his incentive payment (This was one of the very few studies where I asked the agency to send checks after the interview was completed, rather than handing people the incentive directly myself. Mistake? I don’t know). When I followed up with the recruiter about the missing incentive, I heard in some detail how this participant had already called and yelled at the admin staff.

And so it goes.

Last month I spoke at the O’Reilly Design Conference about user research war stories. The video is now online and I’ve also embedded it below.

Here are the slides

Last month I spoke at Interaction 17 about user research war stories. They’ve put the video online and I’ve also embedded it below.

Here are the slides

and a sketchnote by Chris Noessel

The talk features new war stories from special guests Elizabeth Allen and Noël Bankston.

Update: Here’s the text of my talk, in Spanish.

It’s been two months since Doorbells, Danger, and Dead Batteries was released. Here’s links to presentations, podcasts and more stuff related to the book.

The Book

• Buy from Rosenfeld Media
• Buy from Amazon
• Foreword by Allan Chochinov
• Excerpt from chapter 3
• Excerpt from chapter 11
• FAQ
• Testimonials
• Images

Reviews

• Goodreads
• Amazon

Podcasts

• UX Podcast
• Rosenfeld Media
• User Experience Podcast

Interviews

• O’Reilly Design
• UX Mastery AMA
• Rosenfeld Media

Presentations

• Amuse UX (video)

Other

• War Stories archive
• New story by Noël Bankston, told at Interaction 17
• New story by Elizabeth Allen, told at Interaction 17
• Nick Bowmast on war stories
• Susan Dray on war stories

Noël Bankston is a UX Research Lead and Human Factors Engineer at Zebra Technologies, currently living in Queens, NY. She told this story live at the Interaction 17 conference.

“So Jim, what would you like to do for lunch? “My treat!” It was the moment I had been dreading all day, ironic since I am a lover of food. I was trying to sound chipper but I was worn through.

It was 2 pm and I was starving. I was sitting in the cab of a 48’ tractor trailer in Lowell, Arkansas. This was my first “ride along” research trip and I had not come prepared with snacks. I was doing in-depth generative research of the pick-up and delivery process for a freight company and hadn’t known that we don’t have lunch until all the deliveries were completed.

I was also not prepared for the weather as I am from up north and I thought the South would be hot in late May. It wasn’t – it was a constant drizzle and cold. So I was sitting in the cab feeling small and tired in the oversized loaner jacket that the dispatcher had given me. We had been on the road since 8:45 am but I had arrived at the trailer dispatch site even earlier to observe the set-up process. And that should have been fine, because on a normal day, Jim finishes around noon. But today we saw all the exceptions – an unprepared customer, incorrect paperwork, an obstructed delivery dock, and poor routing. As a researcher, it was a gold-mine as I observed where problems occurred and how Jim handled them. But as someone who is mildly hypoglycemic, it meant I was getting hangry. It had been a long morning of climbing into and out of that cab, learning which hand to place where to get the right leverage to pull yourself up as you step onto the step that is only wide enough for half your foot. And I don’t know how many of you have ridden inside of a tractor trailer but it is loud and you feel every bump.

In that moment as I asked about lunch, damp, tired, and hungry, I thought back on the the anxiety I had felt earlier in the day about lunch. A co-worker told me that on his previous ride-along they had eaten a burger from a gas station mini-mart. Even on a normal day that would make me uneasy, as gas stations aren’t known for freshness and hygiene. I knew that this type of research means being available for wherever the subject takes you, but I was really hoping that didn’t include food poisoning.

But at this point, 8 hours from my previous meal and having no idea what part of town we were in, who was I to be picky?

“So Jim, what would you like to do for lunch?”

“I just want a salad. I try to eat healthy.” I gave a huge sigh of relief, accompanied by a rumble of rejoicing from my stomach. It seemed that between the two of us, I would be eating the bigger meal. I found a nearby Mexican restaurant on Yelp. While enjoying the flavor combination of fresh cilantro and lime with nary a fryolator in sight, I realized how I had been making assumptions about “truckers” based on stereotypes rather than letting the research reveal the truth. And those assumptions were also judgments about health and lifestyle. Jim was aware of the health effects of his job and wasn’t going to turn down an opportunity to have a healthful meal, especially when a researcher was paying! One of the reasons truckers eat unhealthy food is both cost and convenience. Truck stops get food fast and are less expensive. Unfortunately, our food system is set up in a way that fresh, whole food costs much more than highly processed, industrially produced food.

I won’t be able to eliminate all my biases or preconceived notions but I can grow in my awareness of them. I have been on many more ride-alongs and other types of research trips since then. You better believe I always have a granola bar with me.

Elizabeth Allen is a UX Researcher at Shopify, an ecommerce platform based in Canada. She told this story live at the Interaction 17 conference.

A few years ago, I was working at Centralis, a UX research and design consulting firm in the Chicago area. One of our clients was a public transportation agency, and our project involved testing the maps and signage within and between transit stations by accompanying participants as they completed realistic wayfinding scenarios to try to get from station to station and find their correct train or bus.

As part of this testing, my research partner Kathi Kaiser and I included individuals with motor and visual disabilities to make sure they were able to navigate just as well as those who didn’t have these challenges. One participant, Susan, was in a motorized wheelchair, and we began our session with a scenario that had us traveling to a station and accessing an elevated platform where she would wait for a train.

Chicago summers can be very hot and humid, and this was one of the hottest of the year. We were all sweating by the time we got to the station even though it was just a short walk from the coffee shop where we met to start the session. Now, this station had no elevator; instead, outside the station was a very long ramp to reach the platform. This was probably the longest ramp I’d ever seen at a transit station — it had two or three switchbacks just to reach the top!

We started up the ramp, and when we were about halfway up, Susan’s wheelchair started slowing down. “Uh oh”, she said. “I think my battery is about to die. I totally forgot to charge it before I went out, and steep ramps like this always make it run out faster.” Sure enough, a few seconds later, the wheelchair slowed to a halt, completely dead.

At this point, we had to make a decision based on what was best for Susan and for the research: do we end the session early, push Susan’s chair back to our starting point, and explain to our client that we would miss out on gathering valuable accessibility insights, or do we see if we can find a power source and salvage what we can of the session? We explained to Susan that we could either end the session or try to keep going, and luckily, she was still excited about the session and was game to push on — literally.

After wheezing our way up the rest of the ramp, dripping with sweat, we got to the platform and found no electrical outlets in sight. The ticket counter was also closed, but after a lot of roaming around we were able to find the lone janitor. We were very fortunate, because he was extremely kind, and offered to let us plug Susan’s chair into an outlet in one of the back rooms.

This story ends happily. After a half hour or so, Susan’s chair was charged up, and during that time we were able to improvise some interview questions and short scenarios we could talk through with her while we waited. It really helped that we were able to think on our feet and that we had a participant who had a positive attitude and was interested in the session. Overall, we were able to salvage a research session that was difficult to recruit for, and our client was really happy with what we learned.

For the latest Rosenfeld Review podcast, I spoke with Lou Rosenfeld about my journey in collecting user research war stories and how that became Doorbells, Danger, and Dead Batteries. (trigger warning: I’m extremely enthusiastic for the first 15 seconds or so)

(embedded below, or go here)

Here’s “Five Questions” with me about war stories, user research, and the O’Reilly Design Conference. Click for the whole thing; Below is a short excerpt.

Why is user research so hard to do well?

I talked about this a bit in Interviewing Users—that the assumption we can just use our social defaults because it’s just “talking to people” holds us back from being better at user research. We have to unlearn a lot of patterns (e.g., sharing about yourself) in order to get to a very different outcome (a good session versus a good hangout). In looking at war stories, I’m digging further into the challenges we face in doing research, and hopefully not stating the obvious but research with other people will have massive elements of unpredictability in it. That means we learn what we didn’t know we didn’t know (and would never otherwise have thought to ask about), but it also means that our attempts to plan and control the process are somewhat foolish (and yet, someone who does research without planning is obviously a fool). There’s an element here of temperament, or worldview, that isn’t so natural for everyone. In some of the stories I’ve gathered, people do everything right, and yet things still go wrong. That’s not a welcome truth.

Last week I did an online chat with UX Mastery about Doorbells, Danger, and Dead Batteries. See one snippet below, but please check out the entire transcript here.

crystal: Do you think that approaching the interview with some small vulnerability of your own allows them to be more vulnerable as well and open up and give more insight? And have you found that added insight to often add value to the research?

steveportigal: our own vulnerability – that’s fascinating and I don’t have a clear take on that. I think a shallow reading says being vulnerable means sharing about ourselves and I am mostly against doing that most of the time for most researchers but it makes me ponder what’s a richer more nuanced sense of what our own vulnerability is, if by being still, present, focused, listening, and not needing to make it about us, we might convey some vulnerability. I think it’s meeting people where they are, accepting them where they are and not putting ourself into it. Which – to your point – feels DAMN risky to a lot of people. Set aside your agenda and listen but do so in a productive effective you’re-on-the-job way, so you are balancing different forces and risks.

I dunno, is that ‘vulnerable?’

Gerry Gaffney interviewed me about Doorbells, Danger, and Dead Batteries (which he contributed to) – and user research in general – for his User Experience podcast.

Check out the podcast on iTunes. Or listen via Gerry’s site (which also features a transcript). Or, listen below.

The idea that some of these ideas are metaphors for life I think is absolutely true and, again, I can sound kind of highfalutin and pretentious here but I think the thinking that I went through in this book is looking at… some of these external factors, right? You know, make sure your camera is ready and you don’t break the cable and you know the sort of “equipmenty” type things that we have to think about. But so many of these are about what do we do when the unexpected happens and acknowledge the unexpected is going to happen and that those are definitely life skills. And I think one of the takeaways that I come back to several times, and I just alluded to it a minute ago, which is know when to walk away. You know and so when you’re in a situation do you keep trying to turn that situation from a failure into a success or do you say “You know what? This isn’t going to work,” and you leave.

My new book comes out today and Lou Rosenfeld has an enthusiastic appreciation and a bit of the back story about the book’s journey.
Read it all here

I dipped into about a dozen of the 60+ field research war stories that make up the bulk of the book. The stories do what stories are supposed to do: engage. And the contributors have been through some experiences that will make you laugh, sweat with fear and discomfort, and—let’s face it—enjoy a bit of schadenfreude. But it’s wrong to see Steve’s new book simply as a compilation of user research war stories. In Doorbells, Danger, and Dead Batteries, Steve comes through: he delivers a broader framework that’s useful for making sense of user research—or, actually, situations with people.

My new book comes out on December 6!

You can pre-order (with a discount) from Rosenfeld Media (or from Amazon). Also coming soon is an audiobook version! Remember, your review on Amazon really helps drive awareness.

————————————-
Doorbells, Danger, and Dead Batteries
User Research War Stories

User research war stories are personal accounts of the challenges researchers encounter out in the field, where mishaps are inevitable yet incredibly instructive. Doorbells, Danger, and Dead Batteries is a diverse compilation of war stories that range from comically bizarre to astonishingly tragic, tied together with valuable lessons from expert user researcher Steve Portigal.

The stories Steve Portigal knits together here have an extraordinary and immediate intimacy, like listening in on 66 researchers’ bedtime prayers. Anne Lamott says there are essentially three kinds of prayers: help, thanks, and wow! Doorbells, Danger, and Dead Batteries covers the whole range, with humor and wisdom.

Dan Klyn, information architect, co-founder of The Understanding Group (TUG)

See more testimonials

Watch this space for a big announcement about the War Stories coming up after Labor Day. In the meantime, I’ve applied to speak about War Stories in design research at SXSW. You can help here (whether or not you are planning to attend SXSW) by creating an account, voting thumbs up for the proposed talk and even adding a comment.

We’ve got three recent stories, all from the Kitchener-Waterloo area: Jennifer’s War Story: Keeping the Lights on in Vegas, Julia’s War Story: For Want Of A Shoe and Susie’s War Story: A Sigh Is Just A Sigh. And coming up next month at Fluxible in Kitchener-Waterloo, I’ll be presenting Epic FAIL: Takeaways from the War Stories Project.

Jennifer Pretti is the Manager of the User Experience Design Team at Christie Digital in Kitchener, Canada.

At Christie Digital, we have a very niche population of users. Opportunities to observe them using our projectors are highly coveted by my UX team. In February 2014, we were invited by a good customer of ours, Staging Techniques, to observe their setup for the keynote address at Microsoft’s SharePoint Conference. The event was taking place at the Venetian Hotel, in Las Vegas, and the keynote speaker was going to be Bill Clinton. Bill Clinton!

Three of us from Christie made the trip: me, Chris (my lead industrial designer), and a software developer, Eric. Although I had conducted many user sessions for Christie before, this was the first time I was going on site to observe a live event setup and I wasn’t sure what to expect. My biggest worry was that, even though we made it clear we were there just to observe, I would be asked to answer a technical question or troubleshoot some problem and not have a clue what to say or do.

Setup was to begin at midnight the day we arrived. Working night shifts is very common for projectionists since it’s the best time to see and calibrate the light as other setup crews are already done and out of the way. The thought of staying up for a night shift wasn’t something I was particularly looking forward to, especially given jet lag was going to make it feel 3 hours later. But I hoped a mix of adrenaline and caffeine would do the trick.

After landing in Vegas, we headed down to the Expo Hall to get our bearings. The scale of everything in Vegas is massive and oversized, and the hall was no exception. It was a gigantic space – at least two football fields long and one football field wide – and it was completely empty and bare. Whatever vision the event planners had for the space seemed hopelessly unattainable in the 5 days left before the show.

When we arrived, big transport trucks were pulling into the hall to start unloading the many tons of equipment that would be needed to run the show. It was clear that they were behind schedule already. Trusses and scaffolding needed for rigging the projectors hadn’t yet been built, so we decided to split up, with Chris covering the first night shift, and Eric and I heading to bed to get some much needed sleep.

Eric and I returned to the site early the next morning to relieve Chris. The first few hours of our observation time were slow and uneventful due to continued delays with the truss work, but eventually things picked up, and soon projectors were being powered on and rigged into position. Excitement peaked when one of the projectors failed to power on. I stood poised to capture an epic story of problem solving and error recovery, but the crew just shrugged, taped an ‘X’ on the top of the projector, and replaced it with a spare one. Even after I got in touch with tech support to help explain the error code (highlighting quite clearly that our error messages need a lot of work), it didn’t change their approach. Time is money and using a functional projector was simply the most efficient option. Whatever the problem was, it could wait until they were back in the office to sort out.

It became clear by the end of the second night that the most interesting portions of the setup would be delayed past our planned departure date. The senior projectionist, Pete, pleaded for one of us to stay a bit longer. I think there was mix of professional pride in his insistence, but (happily for us) a realization of the mutual benefit of our presence, observing their workflows and listening to their wishlists. It was on account of his enthusiasm that I agreed to change my flight and stay an extra night. My fatigued body howled in despair. Another night shift? Are you crazy?!

There is no better place to change your sleeping patterns than Vegas. That city looks the same no matter what the hour: there are always people walking around, always a restaurant open, and enough indoor walkways that it could be any time of day. Hotel rooms come equipped with industrial-strength black-out curtains, whose existence I suddenly appreciated in a whole new light (pun intended), as I tried to convince my body that falling asleep at 10 AM was a totally legit plan.

The little sleep I got left me with major doubts that I could keep up a respectable and coherent state of mind for my last night. However, early into the shift, Pete insisted I help him colour match the displays. Colour matching 26 projectors is a very laborious activity that had us whizzing around on a golf cart, playing with light meters, and debating whether one projector was a fraction more magenta than the other. Shifting from observer to honorary crew member made the night fly by and gave me a more rich perspective of how our products are used.

I didn’t sleep until I was on the airplane later that afternoon. I welcomed the rest, but felt a pang of regret for not extending my trip long enough to see Bill Clinton speak. As social media began to light up with pictures of the event, I cheered for Staging Techniques and Christie for a job well done. And smiled knowing that Bill Clinton was walking on the same stage where I had been, just 24 hours ago.

Julia Thompson is a Design Research & Insights Analyst at BlackBerry in Waterloo, Canada.

It all started with a simple question from the dispatcher: “Do you want a call when your taxi arrives?” My nonchalant answer: “No thanks, I should be okay.” was the nail in my coffin. This was my first error in a series of cascading mistakes.

The next morning I was heading out-of-country for in-home interviews. That night, in an effort to be as prepared as possible, I called to arrange a taxi for an early morning pickup. I hung up the phone and proceeded to pack my bags. I considered carefully what to pack. I visualised my next few days: what would the weather be like? What would be my mode of transportation? What clothing would be appropriate for the work – casual enough to fit into a home environment and dressy enough to fit into an office environment? I was sure that I had considered all the details. Unfortunately, the most important detail, my alarm, was what I missed.

Satisfied with my preparation, I went to bed, and slept well. The next morning I awoke feeling refreshed. With birds chirping outside, sunlight filled the room. Yet something felt terribly wrong. What time was it? Why was it so light out? I picked up my phone, checked my alarm, and then checked the time. My stomach fell to the floor. My flight was leaving now. Sheer panic overtook me. I couldn’t think straight. I had never missed a flight before. I felt like I was going to throw up. I was paralyzed, I had no idea what to do. I grabbed my phone and called our corporate travel agent. It felt like hours as I waited on hold to ask my pressing questions: Could I still make my interview? When was the next flight? Could I fly out of a different airport instead? The sound of my heartbeat drowned out every noise as I sat there waiting, palms sweating, phone clutched. The agent came back on the line and said there was a flight leaving from another airport in 2 hours. Could I make it there in time? It’s almost rush hour. It’s an hour’s drive with no traffic. What about parking? Customs? Security? If I took the car, how would my husband get to work? On top of all that the agent still wasn’t sure whether there was room on the flight. We decided, together, that I should start driving and I should stay on the line while she called the airline to confirm availability. I jumped in the car, with my phone on the passenger seat and that awful music taunting me as I continued to wait, on hold. I got about 10 minutes down the road when the agent told me to pull over and go home. That flight wouldn’t be mine. I would settle for another flight, hours later, and hours after my scheduled interview.

Later that day, as my plane came in for its landing, I just felt low. I was tired from the emotional rollercoaster of missing my flight, I was anxious knowing I’d have to tell the people I was working with what had happened and I was sad that I had missed out on an interview and the opportunity to see, first-hand, into the life of one of our customers. The only thing saving me was the fact that I was the client and so, even though I missed the interview, it still went ahead as scheduled.

The following day I awoke, in the right place and at the right time, with a better perspective on life. Our local research partner was gracious enough to include me in an interview that day. I was thankful. I was relieved. But now, that meant there would be four of us attending this interview. Two consultants and two clients; two too many. The consultant had called ahead and confirmed with our interviewee that it would be okay if an additional person (me!) attended the interview. Our interviewee was very accommodating and agreed to have all four of us into her home. I was so preoccupied with resolving my own error that I didn’t consider, until later, how the dynamic of the interview would now be affected.

We all got to the interview, we all walked in, we all sat down in the chairs offered to us by our interviewee. As everyone was setting up I started to look around and take note of the environment. I noticed several pairs of shoes neatly arranged by the front door. I looked over at our host, I looked down: bare feet. My eyes darted around the room, I looked down at all our feet. All four of us had our shoes on, laces tied. Bah! We were the worst guests ever. Weren’t we all, as researchers, supposed to notice something so simple but so important?

I spent the next five minutes cursing myself, my missed flight, the totally wrong and overpowering dynamic of four researchers to one customer, and the miss on basic shoe etiquette. I had to shake it off – all the feelings of shame, all the feelings of doubt – and I had to focus. I had to be in the moment, I had to get the most I could out of the interview and I had to show the interviewee the respect she deserved.

It ended up being a great discussion. It was, by no means, a textbook in-context interview, but we had a nice dynamic emerge nonetheless. My story is not one of a single epic fail, but instead of a series of errors with a cascading effect. “For want of a nail the shoe was lost, for want of a shoe the horse was lost…” Here, we had not a want for a shoe, we had too many.