Patricia’s War Story: The Hidden Persuader
By Steve Portigal at 8:30 am, Friday October 10 2014
Patricia Colley is an experience designer and the Principal of Creative Catalysts in Portland, OR.
In 1984, I was 23, and working for a market & social research firm in San Antonio, Texas. They sent me down to McAllen to collect voter opinions on the upcoming national elections. McAllen is a sleepy little town near the bottom tip of the state, just a few miles from the Mexican border, mainly populated with low-to-moderate income Hispanic families.
I was on my second day of door-to-door polling, asking voters their opinions on policy matters, and their thoughts on the state and presidential candidates. The work was progressing well. As usual, I was getting a high rate of interview completions, with lots of useful data. After four years of working in market and social research, I was quite confident in my neutral, non-threatening “aw shucks, I’m just one of you” act, and its ability to deliver great results.
But my confidence was shaken when I met Maria, a shy housewife in her early 30’s.
It was about 4 pm on a warm, dry Thursday afternoon when I knocked on the door of a modest, well-kept ranch house in a suburban section of McAllen. Maria opened the door part way. She was half-hiding behind it, sizing me up like a rabbit peering through tall grass at a coyote in the distance…curious, but poised to flee.
Me: “Hello, my name is Patricia, and I’ve been sent here by (XYZ Research) to gather public opinions on the upcoming elections.”
Maria: “Oh, hi.”
Me (turning on the charm): “May I ask you some questions? Don’t worry, I’m not selling anything!”
Maria: “Uhh, sure, I guess?”&
Me: “Great, thanks! This won’t take long.”
Wide-eyed, Maria flashes a shy smile before her jaw slacks again. This one’s cagey, I thought to myself, but I’ll get her talking.
Me: “Now, thinking about (Candidate X), what comes to mind?”
Maria: “Uhh, I don’t know? Is he a good guy?”
Me (shrinking): “Well, I really don’t have any thoughts on (Candidate X). Besides, my bosses didn’t send me all this way to talk about my opinions. He wants to know your opinion.”
Maria: “I don’t know. He seems okay?”
Now, I don’t think Mary is incapable of forming opinions. I suspect she’s simply never been asked to share her thoughts about such important things, so far from home. And she may never be asked again. But on this day, I was determined to make her opinion count.
Me: “Well, you’ve heard of him, maybe seen him on TV?”
Me: “So, what did you think of him? Is he someone you would vote for?”
Her eyes darted across my face, scanning every crease and twitch, searching for clues. Those big rabbit eyes begged mutely for help. I stared back, apologetically. I took a few slow breaths, trying to ground us both, so she might relax into talking more naturally. Each time she hesitates, I carefully repeat the question, altering the wording and inflection to make them sound as simple and benign as possible.
Me: “Really, we’re just interested in what you think. Whatever you think is fine. Do you think you’ll vote for him, or not?”
Maria: “Uh…yes?” (seeing no reaction from me) “No?”
Me: “Okay, that’s fine. Alright. Now, thinking about (Issue A), is that important to you? Do you think it’s good or bad?”
Maria: “Uhh…I think it’s good?”
The back and forth went on for several minutes. I’m trying to go completely neutral and void of any emotional expression, but my contortions only intensified the awkwardness. The interview was in free-fall. I was failing miserably to collect any genuine responses from Maria. A hot wave of panic washed over me. How can I get this back on track?
In that moment, I just had to let go.
I quit fighting it, and fell back on connecting with Maria as a person. As Maria answered my questions, I began riffing on her responses, affirming and adding detail to them. While trying not to reveal my personal opinions, I offered supportive words and gestures to elevate everything she said, so that she might open up and elaborate. Eventually, she did relax, and her answers flowed a bit more freely.
Me: “So, what about the presidential candidates?”
Maria: “I guess I’ll vote for (presidential candidate B).”
Me: “Great! Is it because he is for (issue B)?”
Maria: “Oh, that’s good. Yeah, (B) is good for us.”
Although Maria was warming up to me, I felt I was way off book. It seemed impossible not to sway her answers. Whatever I wrote down, I feared it might be swept away by the slightest shift in body position, or an eyebrow lift. Well – at least she was talking, I told myself.
Finally, we got to the end. Walking back to my car, I breathed a huge sigh of relief. The hardest interview I’d ever done was over. I went out for a well-earned drink and a tragicomic debrief with my co-workers.
Sometimes you just get a dud subject, and it is what it is. But something about that 15-minute exchange with Maria struck a deeper chord in me. As I drove out of town, troubling questions lingered. What is the value of a skewed interview? Was this the only time I’d failed to be impartial? Or, had this been happening all along, in more subtle ways? How can I ever know that the data I’m collecting is pure?
Maria taught me two important things that day.
1. People make stuff up as they go along. And, we can’t always see the flaws in self-reporting.
2. The observer effect is unavoidable. Interviewers shade their work in unpredictable ways.
I’m as diligent as ever about delivering valuable insights through my research. But ever since that incident in McAllen, I draw my conclusions with a fuzzy border, in humble deference to flawed inputs and shadow projections, on both sides of the clipboard.
Tags: fieldwork, patricia colley, war stories
David’s War Story: Let it Bleed
By Steve Portigal at 10:34 am, Monday July 07 2014
David Hoard is an interaction designer and here he shares his second story.
Years ago we were re-designing a device to cool a patient’s blood during open heart surgery. This protects the body during the procedure. The client arranged for us to witness a heart operation, and we were pretty excited about that. My only concern was that I would faint from seeing blood.
Research day came and we headed to a nearby hospital, prepared to be serious, professional researchers. A nurse helped us gown up and get ready. I was expecting the operating room to be a sober technical environment, and I saw that was true. The equipment was stainless steel; the walls and floor were blue-green tile. I anticipated that this would be an orderly collection of findings.
But as soon as the surgery team started to come in, the vibe changed. The nurses chatted. The anesthesiologist joked. The patient, a man in his late fifties, was casually whisked in on a gurney.
The nurses chatted with the patient as they put on the anesthesia mask and he drifted off to sleep. They slathered him with a brown antiseptic wash. It made his skin look like a basted turkey, and I thought “He’s just another piece of meat to them.”
Then things really got started. The surgeon came in and straight away had the nurse hit the music. The sound of the Rolling Stones filled the O.R. The jokes and banter increased. The technician operating the blood cooling machines set to work and we tried to stay focused on that. But it was futile.
When the patient was sufficiently chilled, they set to work with a powered saw and cut open his sternum. They were ripping a person’s body open, and they did it while talking about sport scores.
They pried the chest cavity open and prepared for a bypass procedure. They took a vein from the man’s leg that would be used as a new artery for the heart. “How you doing back there?” came the question from the surgeon. “Good!” we replied, and I realized I wasn’t woozy at all. It was all too fascinating.
It was at that moment that the most surprising thing happened. The surgeon said “How do you like this?” as he put his hand down in the chest and lifted the beating heart up and out. The music thumped, the heart pumped and the surgeon gave us a wicked grin. He knew full well he was holding the patient’s life in his hands. But at the same time, it was all in a days work for him. No big deal.
After completing the bypass, they finished their work and stapled the man up. The surgeon cleaned up and zoomed off to something else important. Before we knew it our research session was over.
As for our actual goal of observing blood-cooling machine, we did gather information about that, but the bigger lesson was in understanding the true nature of our users. We expected one-dimensional experts and we saw three-dimensional humans.
My work on projects like this has taught me that experts are simply regular humans with a specialized job to do. Help them be smarter, help them be more successful. But don’t forget the human underneath that needs ease of learning, ease of use and help preventing errors. Humans don’t want to devote 100% of their brainpower to your product. They need to reserve some for cracking jokes and singing with the music.
When your research goes in an unexpected direction, go with the flow and let the Stones play. You might learn something more meaningful than your original plan.
Tags: david hoard, fieldwork, heart, operation, surgeon, surgery, war stories
Ari’s War Story: Chicken Run
By Steve Portigal at 9:06 am, Tuesday July 01 2014
Ari Nave is Principal at The King’s Indian.
My very first field research was in the north of Ghana along the Volta River north of Keta Krachi, trying to unpack the usage rights and other factors that enable the sustainable use of a common pool resource (in defiance of the tragedy of the commons).
The research was hard. I was isolated, lonely, and physically drained. No one in the village spoke English. They spoke primarily Ewe https://en.wikipedia.org/wiki/Ewe_language and I was communicating through an interpreter. I had a feeling that I was missing a lot of nuance and detail with the interpreter and had several discussions with him about my concern.
I was also sick as hell of eating fish stew with fufu or gari. For one thing, it was spicy as hell…so spicy that at every meal I had these convulsive hiccups. This hilarity may have endeared me to my host, but the diet was monotonous.
I had spotted guinea fowl wondering around the village. I asked my host family about it and they just laughed and said they are wild animals.
So I set my mind to catch one. That evening I watched as the guinea fowl hopped up a tree in the village. They used the same tree each night and seemed to jump up in a predictable pattern.
The next evening I was prepared. I had a long string for my trap. I tied a slip knot on one end and placed the snare on a protrusion of the trunk that was chest-height, a pivotal step on their journey up the tree.
The string was about 50 feet long and I ran the length straight to another tree that I hid behind.
The folks in the village just laughed at me, which they seemed to do with great frequency. But I was determined. Patiently, I waited.
As dusk fell the fowl made their way up the tree. When the third bird was on the spot I yanked as hard and fast as I could, while running in the opposite direction. And I had the little bastard. He flapped his wings and I reeled in the string, and soon had a plump guinea fowl in my hands. My host and all the other villagers came running at the commotion and now stood with jaw agape as I proudly displayed my bird.
I asked my host to put the bird in a basket and put a big rock on top to keep him secure. It was too late to cook them so I ate my mind-alteringly hot fish stew but with a content mind, thinking about the fowl I was going to eat for dinner the next night.
I woke up refreshed and optimistic. I gathered up my notebook, camera and tape recorder and headed out, but first stopped to gloat at my catch. To my dismay, it was gone. I shouted and my host came running over. “He has escaped in the night,” he explained by way of my interpreter. No way, I thought. The boulder was still on top of the basket. Someone stole my bird. When I voiced my opinion to him he shook his head and simply repeated the claim.
That night, I executed my hunt again, with equal success. This time, a larger group came out to watch my escapades and were equally surprised both by my technique and success. Again, I place the bird in the basket, this time adding another large rock on top.
The next morning, I woke with foreboding. I jumped out of bed and checked the basket. Stolen! I was pissed off. My host tried to placate me but I was having none of it. Arrogantly, I told him that I was going to complain to the head of the village. My host shook his head. He waved to me to follow him.
We walked toward the center of the village where the elder lived, ironically where the guinea fowl often congregated. Before we reached his compound, my host swooped down and picked up a guinea fowl with his hands! Of course I had tried this many times when I first got the notion to eat one, but ended up running around like a fool. He lifted the wing of the fowl and I could see a colored ribbon. “Each bird is owned by a family,” he told me. “There are no wild birds here.”
So I had captured a bird that was someone else’s property. I was confused as he had earlier told me they were wild animals. In the end, it turned out that he never thought I would be able to capture one, nor did he understand why I wanted to capture one. When I explained that, while I loved the fish stew, I wanted to expand my eating horizons, he laughed. “Just buy one from the neighbor and my daughter will cook it for you.”
So that afternoon I bought a fat guinea fowl and the daughter of my host prepared the most delicious ground-nut stew with him. To this day, I crave that stew. It was unlike anything I had before and better than anything I could have imagined. Although, it was still insanely spicy.
I felt a bit idiotic about the entire episode and it only reinforced to the folks in my village how odd I was. But it had one positive side-effect. People realized how little I understood about even the basics of their lives, and they began to be much less assumptive about my state of knowledge.
Note: A similar recipe is here.
Tags: ari nave, chicken, dinner, fieldwork, food, ghana, guinea fowl, meal, spicy, war story
Carol’s War Story: Driving Force
By Steve Portigal at 2:54 pm, Wednesday June 11 2014
Carol Rossi is the senior director of user experience research at edmunds.com.
Since Edmunds.com is an auto website we spend a lot of time hearing about how people shop for cars. A couple of years ago we ran a shop-along study where we conducted in-home interviews to both understand car shopping behavior and simultaneously screen people we may want to go with on test drives to dealerships. I always take someone else with me when running interviews – a designer, product manager, exec, etc. – so they get first-hand exposure to real car shoppers.
This time I had the head of editorial with me. The Edmunds editorial team has a long-term fleet of cars so they can write about car ownership. My colleague tells me that he’ll drive and we’ll take one of the fleet cars. We meet in the lobby and he walks us over to a $100,000 red BMW. Not what I typically show up in to interview somebody who is probably shopping for a Honda.
The interview is in Hollywood and although it’s only 10 miles from our office this is LA so we drive up Santa Monica Blvd for like an hour. We find the address and it’s not in the best part of Hollywood. There we are with this six-figure car. Eventually we find a parking spot that looks relatively safe and walk to the building.
We use the callbox and are buzzed into the building. We look for the apartment and realize it’s in the basement. We’re greeted by our interviewee, a middle-aged guy who’s described on the screener as a self-employed writer (like much of the population of Hollywood). The apartment is the tiniest living space. It really looked more like a one-car garage. The air was stuffy, there was a unique odor that was somewhere between musty and dusty, there were no windows open and no A/C, with carpet that had maybe never been cleaned. I started to hope the allergy attack I was sure was coming happened after we were finished. The apartment was overstuffed with piles of papers (screenplays?), VHS tapes, and posters of independent movies (including one with a woman in bondage gear who we later discover is his wife). Although we’d normally want to capture anything descriptive of the scene, to avoid distracting the product team who would watch the video later we had to position the camera to keep the poster out of the shot.
We’re chatting and after a few minutes our interviewee’s 35-year old wife comes out with a baby. The wife is some kind of Hungarian model (think of a European version of Gisele Bündchen). The guy turned out to be really nice, educated and articulate, but also clearly not at all someone likely to test drive a car at a dealership. Basically he hates cars, rides his bike everywhere, is trying to get off the grid but needs a car now that there’s a baby, and says he’ll buy some used car that’s parked on the street with a sign in the window.
Was this interview all for naught? From the first moment through the end I wasn’t sure. You always learn something new, so even though this guy did not meet our criteria for people likely to buy a car at a dealership we certainly got exposure to a type of shopper we knew theoretically existed but hadn’t yet encountered (“the eccentric car hater”).
I’ve seen homes like this (and worse) but after the interview we walked outside and my colleague couldn’t unload fast enough. He’d never seen a living situation like that. In rapid succession he declared (out of concern for our safety) “When we first walked in I though it was a trap – I was looking for a way out” but then (out of concern for the child’s health) repeated several times “They have a baby in there!!” And then he began to express his concern for my safety “Do you go on these interviews alone?…You take a guy with you, right?”
After this emotional decompression, we jumped back into the ostentatious Beemer and drove down Santa Monica Blvd., away from the unknown of the ethnographer’s life to the predictable comfort of our office…until the next interview.
Tags: beemer, bmw, carol rossi, cars, fieldwork, gisele bundchen, war stories
Rachel’s War Story: Research, in Sickness and in Health
By Steve Portigal at 8:12 am, Friday May 23 2014
Rachel Shadoan is co-founder of Akashic Labs, a research consultancy that leverages hybrid methodologies to create rich and accurate portraits of users.
It was my first field assignment out of school. Okay, technically it wasn’t my assignment–a contractor would be conducting the interviews, and I would be along to observe and record. But I’d spent the previous two years and six months in a lab writing code, so I would take what I could get. To say that I was excited would be an understatement. I was stoked.
Plus, I’d get to fly to California! I’d be on an honest-to-goodness business trip! It was going to be great.
It certainly started out great. In the shuttle from the airport to the hotel, I counted citrus trees. Citrus trees! Growing in the ground! In people’s yards! And no one seemed shocked by this! Of course, I had plenty of time to count those trees, as we crawled through traffic for hours. But the weather was glorious and I, like it, was ebullient.
Things began to look dicey, however, when I met the researcher I would be working with. She was a smart, gregarious woman, who also happened to be sick. Very sick. Down-a-bottle-of-Nyquil-and-sleep-it-off-for-a-week sick.
Rest and recuperation, unfortunately, were luxuries we could not afford. The project was on a tight timeline and already behind. At least one of the interviews we had planned had been rescheduled once. Stakeholders across three organizations were chomping at the bit. It was, in the melodramatic way of business schedules, do or die.
And so we did. We pre-gamed with Thai food, guzzling tom yum soup for its sinus-clearing properties before returning to the hotel for an early-to-bed. The following morning we set off, my compatriot fueled by a powerful cocktail of cold medicine and espresso, myself running mostly on nerves and the delicious feeling of being free of my cubicle confines.
Still, we felt uncomfortable bringing sickness into the homes of our participants. “Give us your insight, and we’ll give you the plague!” is not the most enticing slogan a researcher could come up with. We tried to minimize risks. I shook hands with the participants; she abstained. She positioned herself as far away from them as their living rooms and rapport-building would allow, with me, a human note-taking buffer, in between. We strove not to be vectors of disease.
Given the circumstances, the first two interviews went well. But after hours of driving hither and yon across the north Bay Area, in traffic that I would have avoided navigating even with a clear head, my partner’s energy was flagging and the cold medicine wearing off. She tossed back an emergency booster of DayQuil in a Starbucks parking lot and we steeled ourselves for the final interview. It was perhaps more disorganized than the first two interviews, but we muddled through together.
And then, as the sun sank below the side of the endless freeway, it was over and we were once again untroubled by the inflexibility of a corporate system that put us in the ethical quandary of whether to conduct field work–or work at all–while ill. We parted ways at a BART station. She headed home to collapse into a restorative, cold-medicine induced coma; I went in to the city to spend a few days basking in the glow of more-or-less-successful fieldwork.
My basking didn’t last long, of course. In no time at all, I had a cold.
Tags: cold, ill, Rachel Shadoan, sick, war stories
Jon’s War Story: Of Speed and Strip Clubs
By Steve Portigal at 8:56 am, Thursday May 22 2014
Jon McNeill is the Principal of Hunter.
Relatively early in my career, as I began stepping out and leading studies on my own, I was in Miami Beach doing ethnographic interviews with participatory “drive-alongs” for a luxury car brand. It was the last day in town, and I, with client in tow, had three 3-hour interviews scheduled that had to get done before we could fly out in the morning, the last one being scheduled for 9pm. This last interview was with Kenny, a guy who was actually supposed to be interviewed earlier in the week, but had to cancel because his yacht broke down and he was stranded for the day on a small island off the coast. We hear a lot of different excuses for non-participation, but that was a new one.
My client and I get through our first two interviews that day at around 8, hop back in the rental car, and start the trip to interview 3, feeling hungry and tired, having missed dinner. I called Kenny to confirm that we were coming, in case he was on another island. He answered in an energetic but distracted tone: “Yeah, laying out the drinks right now. We’ll get in the car, go get some speed, and come back and I’ll give you whatever you need.” Click.
“Speed? Oh no. Who is this guy? He must mean going fast, in his car,” I thought to myself.
I warned my client that we might have a live wire on our hands, but that we’d just go get the interview that we needed and then grab a bite.
We arrive to the address to see Kenny out front, waiting for us. “My wife is putting the kids to bed right now,” he told us, “so I’d rather not go in just yet and disturb them. Why don’t we get in my car, do the drive, go get a beer, and then come back and do the interview thing?”
We usually did the drive-along as the last part of the interview, but as intrepid researchers! Going with the flow is what we do best! Plus, at this point in the day, a drink sounded pretty good. My client and I nodded our agreement and squeezed into Kenny’s convertible: me riding shotgun, and my client folded into the tiny backseat area, holding the camcorder.
As soon as I buckled my seatbelt, Kenny hit the gas and I saw the speedometer jump up to 110 mph. I looked back at my client, white knuckled and – like a champ – rolling video on the whole thing.
We rocketed through a number of dark, mostly empty Miami streets. I was disoriented but loving the way the car gripped the pavement as we took turns in high gear. Just as I was wondering why he was choosing to take us to a bar that was so far from his home, I noticed a police cruiser waiting at a stop light ahead of us. Either Kenny didn’t notice, or he wasn’t worried; we flew through the intersection, still doing over 100.
I flashed on how the rest of the evening might unfold: sirens, mug shots, bailing my informant out of jail… but the cruiser didn’t even give chase. I think the officer knew he wouldn’t catch us.
Finally we pulled into a large parking lot, full of expensive cars, in front of a small oblong building. Two huge bouncers stood out front.
Kenny turned to us and said, “Welcome to the best all-black strip club in Miami Beach!” and headed for the entrance before I could fully process what that meant. My client’s mouth was agape.
Neither my client nor I are what you might call “strip club people”. He had been telling me about how he and his partner were remodeling their house into a real mid-century modern masterpiece. As I looked down at myself, I saw with dismay that the polo shirt I was wearing kind of made me look like the guy on Blue’s Clues.
Since this experience, I’ve heard stories of researchers obliging their clients by taking them to strip clubs, all in the name of client services. And Miami’s relationship to strip clubs did seem to be more casual than other parts of the country, because a few of our other participants had mentioned in passing eating lunch or getting a drink at a strip club. But I was mortified – this was not something I was anticipating. Yet at the same time, I felt cuffed: I knew we had to get this interview checked off, and I didn’t feel like I could demand that we return to his home without ruining our chances at building strong rapport.
I turned to my client and said, “I am so sorry. If I had any idea that he was taking us here, I wouldn’t have agreed. But at this point, I’m worried about insulting him; so let’s just go in, have a quick drink, and head out.”
My client, a saint, shrugged and said, “This is just what happens when you do ethnography, right?” Right.
The bouncers patted us down and we walked inside. Not having a depth of experience in this area, I had to take Kenny’s word for it being the best of its kind in Miami. Kenny was already at the bar, waiting with our drinks.
“So, what do you want to know?” he asked me, as he handed me a beer.
I struggled to remember my protocol questions, and we talked for about five minutes before Kenny excused himself to go to the bathroom. I looked over at my client and we both made a silent acknowledgement that we were done with our beers and ready to go.
Just then, Kenny came back with a stripper on his arm. He turned to my client: “Hey, I bought you a lap dance.”
My client’s face went white. The room began to spin. My client tried to politely decline.
Kenny, confused, said, “No, she’s great, I’ve had her before!”
My client politely declined again, and suggested Kenny go for it.
Kenny asked him, “What is it? Are you married?”
“You have a girlfriend that would disapprove?”
“Well, then, what is it?”
My client started stumbling over his words, trying to come up with a firmer excuse. Then Kenny laid down his trump card.
“Look, man, I’m doing this because everyone thinks you’re cops. You’re white, clearly not having a good time, and if you don’t do this, they’re probably going to take us outside and beat us up.” He waited for my client to answer.
My client looked at me the way survivors of a shipwreck must look at the person holding a life preserver. To my shame, I looked away.
My client, resigned, was led back to a private room. I turned back around in my seat and started processing all that had happened: my conversations with my client, some of the things he said that I hadn’t caught at the time, his answers to Kenny just then… and it all suddenly clicked for me, with a sickening certainty.
Kenny handed me another beer and said, “You know, I think your colleague might be gay.”
“Yeah,” I told him, “I just figured that out myself. But what you don’t know is he’s actually not my colleague, he’s actually my client. You just gave a private lap dance to my gay client.”
I felt ill. Kenny started laughing.
“That’s really funny, man. That’s really funny.”
I think Kenny really felt badly about the whole thing. After my client returned, we left and Kenny took us out to dinner at a kitschy piano bar owned by an old gay friend of his. We all laughed and told stories about crazy things that had happened to us in our lives, and at the end, without us knowing, Kenny paid for everything.
The night ended back at Kenny’s house, in front of a literal parking lot full of his Audis, Porsches, and huge SUVs. He was a fantastic informant, and helped me craft the recommendations for the brand based on his interview.
The car ride back to the hotel was pretty quiet. “Strange night, huh.” I said. My client nodded his head.
We shook hands at the hotel elevator and said goodnight. That was the last time I saw him – he wasn’t at the final presentation, and I heard that he had left the company not too long afterward.
At the end of the study, we sent him a client satisfaction survey, which was standard practice for us at that time. To my shock, it came back straight 10s. My client was a saint.
Unlike many of the other War Stories, this doesn’t paint me in the best light – mistakes were made, character flaws became apparent. But in some ways, the ability to realize that you’ve made mistakes and are flawed is one of the things I treasure most about anthropology — ever since my Intro to Anthro college courses where I began to learn about the long, illustrious line of mistaken and flawed anthropologists who came before me. In fact, often those mistakes and faux pas were the keys to unlocking some heretofore hidden cultural truths. And I think that night was no different, although I don’t think the cultural truths that were unlocked for me were necessarily about luxury automobiles.
I can’t see myself getting into the same situation now – there were at least two inflection points that night where today I would have directed things differently – but it could be that going through that experience together, the three of us, led to a deeper connection and (eventually) a successful interview. It certainly led to a War Story.
Tags: fieldwork, jon mcneil, lap dance, luxury automobile, miami, strip club, war stories
Doug’s War Story: Knock-knock! Who’s there?
By Steve Portigal at 8:32 am, Wednesday May 21 2014
Doug Cooke is founder of Tinder, a research consultancy focused on people-centered innovation.
In a recent research and strategy project focused on defining a new global platform for a medical device, our research plan required us to shadow clinicians and others as they used existing devices in the “context of care.” With minor issues like HIPAA protecting patient privacy and other security issues at big urban hospitals in the US, our team decided that conducting research in Europe provided a better opportunity to understand these devices and their users.
Planning started with all the usual steps: multi-day client sessions to assess the domain, issues and problems; auditing reams of client data and documents; becoming familiar with competitive products, etc. We developed a research protocol that went through many rounds of revision with a large, multi-location client team, arriving at a clear understanding of relevant and important user issues. We developed screening criteria for participating medial institutions. Pilot studies were run at US hospitals. Months of preparation were spent in making sure our research team was fully prepared to bring back insights and perspectives that would help define the next generation global respirator platform. Ready, set, on to Europe!
Our first stop was a hospital in Wales. They had lined up the appropriate people for us to shadow and interview, including department heads, physicians,and medical techs. We spent two days shadowing, probing and gathering, and everything worked according to plan. Wahoo!
At our second stop in London (hauling two large model cases that would not fit into London’s very spacious cabs), we arrived at the check-in desk and ask to see Dr. Smith (or so we’ll call him). Upon arrival at his department wing, we learned that Dr. Smith was not in. Even more concerning was that Dr. Smith was out of the country at a conference and had not let anyone else know we were coming. After speaking with a few more people, the answer was “Please come back at another time when the doctor is in.” Ouch! In spite of all the planning, effort, and resources to get here, a few uncooperative people were about to jeopardize our research program.
How could this happen? Well, I ignored one of my primary rules: never let the client take on a critical path item that could endanger the project’s success and my firm’s reputation. Specifically, because of the difficulty of gaining access to the right people and institutions, and extremely high cost if we were to use a traditional recruiting process, our client took on the responsibility for arranging our visits to hospitals through Europe. Few clients understand the level of effort needed to screen, schedule and triple-confirm each participant. When the “research gig” is complex and requires the participation of a number of people carefully choreographed in a short time, it is essential to have a dedicated, experienced resource to make that happen.
We made it all work in the end. With no Dr. Smith and an apparent dead end, we literally started on-the-spot networking, walking up and introducing ourselves to doctor after doctor until we had made some friends that would grant us two days of access in the ICU and ER. It worked out in the end, but presented unforeseen delays and stress to an already pressure-filled project. Painful but constructive outcomes, nonetheless.
The rest of the trip in Germany and Italy presented various levels of preparedness on the part of hospitals we visited. Some hospitals were planning on hosting us for our full two day itinerary and some were expecting only a few hours meeting (which we were able to extend by turning on our best charm).
I have always been a very careful planner and can fastidiously orchestrate research logistics. I know what it takes to gather user insights. But the lessons learned from this European research foray is a clear reminders that whenever I can, I must control the recruiting and scheduling process. I hope to never again knock on any unsuspecting doors.
Tags: client, doctor, doug cooke, recruiting, war stories
Chauncey’s War Story: Secrets, Security and Contextual Inquiry
By Steve Portigal at 8:35 am, Friday May 16 2014
UX architect Chauncey Wilson shares a rather scary story about permissions gone missing.
In the 1980s, I worked for about 7 years at Digital Equipment Corporation (DEC) as a usability engineer. My group was led by John Whiteside, who pushed to make usability a serious discipline informed by metrics, fieldwork, and lab studies. The method of contextual inquiry was developed in our group by John, Karen Holtzblatt, Sandy Jones and Dennis Wixon. We did a lot of fieldwork to refine our methods and inform product teams about how to improve their products.
During my tenure at DEC, I set up a set of interviews with a major client who must still go unnamed. The client did military research and used some of our products. I got clearance to interview people at the site with the caveat that all videos, tapes, and notes would be surrendered when I left. I would analyze the data at their site and do a presentation about my findings, leave all data, and not discuss any details of my interviews. I got to the site early in the morning and signed in at the front desk. In those days, we had 8mm video cameras as our primary tool for field interviews. I had permission from the senior security chief to videotape the screens and record sound for 5 different users of our DEC products. I started setting up my equipment for the first interview and about the time I got to mounting the video camera on a tripod, three really large security guards with weapons blocked the exit to the office and asked me what I was doing (“I’m here doing some research for DEC”), then they grabbed my equipment and took me to a holding area and proceeded to interrogate me. I said that I had sought permission and had an agreement with the chief security officer – but that agreement was not to be found.
My name had been on the visitor list and the people I was interviewing vouched that I had set things up with them, but there was no clear approval for videotaping. I asked if they could contact their security chief, but he was on a vacation in the Virgin Islands. While they called and left messages for him, I spent a few hours in the holding area (you might call it a “cell”) concerned that I might go to prison. Though it took a while, they did catch up with the security chief and took me back to the cube where I had started my set-up and let me continue.
I spent a week at this site and noticed that the guards walked by and checked in on me a lot. Every night when I left during the week, they had me empty my pockets and remove every item from my briefcase. On Friday, I put together a report and presented to an audience of very serious people who asked no questions. I left all the data, submitted to my final contraband search and left the most bizarre field visit of my entire career.
Tags: chauncey wilson, search, security, war stories
Gerry’s War Story: Right to be Wrong
By Steve Portigal at 8:12 am, Friday May 09 2014
Gerry Gaffney runs the UX consultancy Information & Design in Melbourne. He publishes the User Experience podcast and is current director of publications at UXPA.
I was researching, with my colleague Patrizia Bordignon, how people thought about and dealt with home renovations.
One of the methods was a diary study (“cultural probe”), and we had carefully recruited – or so we believed – a small set of participants with whom we would work for several weeks.
Warning bells sounded fairly early with one of the participants, who showed up very late for the initial briefing. These things can happen, so we ran a separate briefing session for him, gave him his kit of reporting materials (camera, diary and so on) and sent him on his way. Let’s call him Mr. W.
Three days after the briefing we telephoned each of the participants. It’s a good idea to do this to remind people about their commitment, to redirect as necessary, and to address any issues that arise. All our participants were on-track, with the notable exception of Mr. W, who seemed somewhat evasive in his answers.
At the end of the first week, we visited the participants. Again, this is good practice; it’s an opportunity to see how the data is being gathered, and what changes might be needed to the process. We also use that opportunity to make a part-payment to the participants, which can serve as a nice motivation.
We were delighted with what we saw. Participants had kept bills and receipts, photographs and magazine clippings, they showed us their renovations or their plans, and we were confident that we were getting plenty of highly relevant data.
When we visited Mr. W’s house, however, it was evident from the first moment that his home was different. The front gate didn’t work properly and the hinges squeaked, the garden was unkempt and the house gave an overall sense of dilapidation. Inside it was a similar story. Every room was in dire need of immediate restorative work, but none was evident. I felt a tad depressed as we drank tea from cracked mugs and listened to Mr. W list the things that needed to be fixed.
Mr. W was not an enthusiastic renovator. His house represented a series of urgent and necessary tasks, none of which had been tackled.
It looked like we would collect no useful data from Mr. W, and as we traveled back to the office we talked about our disappointment and reexamined our recruiting strategy.
However, as we moved into data analysis, we found ourselves referring quite often to Mr. W, and gradually came to realize (no doubt this should have been obvious earlier) that Mr. W’s world was in fact directly relevant to our project. While the enthusiastic renovator was undoubtedly a key consideration, the unenthused or reluctant could also present great opportunities. Their needs and goals were different, their attitudes were different, and the way that we would design for those characteristics was different.
In many ways, in fact, Mr. W was an ideal participant specifically because he didn’t fit our expectations. He challenged the underpinnings of the project, and made us examine our design decisions in a much more rigorous fashion.
I often reflect back on this experience when I’m doing user research, and I specifically watch out for negative reactions and experiences, because they can often teach us things that we might not otherwise learn.
I still believe it’s important to recruit carefully, but perhaps we should be more open to the idea that the “wrong” participant is sometimes precisely the right one.
Tags: fieldwork, gerry gaffney, recruiting, renovation, unenthused, war stories
Steve’s War Story: Finding Mojo “In the Moment”
By Steve Portigal at 7:23 am, Thursday May 08 2014
Steve Sato is the Principal at Sato+Partners, a customer-centered strategy and stakeholder-centered organization design consultancy.
We were three days into our 18-day research trip. The clock was ticking and our progress had been frustratingly slow. We had nary an insight to show for our time spent here so far. It was 9 o’clock in the morning and we were already hot and sweaty after having walked a quarter of a mile on the footpath, the only way to a remote village in Uganda. Our team was doing field research on making microfinance more efficient and reliable, so banks and other financial institutions would find it profitable for them to extend their services to include microfinancing. The current system of paper and pencil, traveling back and forth to an office two hours away, and then transcribing notes onto a PC (“sneaker net”) was inefficient and fraught with errors and omissions. Furthermore, what was required was not only an IT system that could span “the last mile” but we had 15 days left to prototype an interaction model that would augment the device. It needed to be a process that the field agents and their clients would trust and adopt without much help. On top of that we had to identify what other not-for-profit and for-profit organizations (e.g., medical, agriculture, manufacturing and so on) would find the field device useful (so we could size the potential market for the device).
I was responsible for the research and the results. I really was feeling the stress and the jet lag and I had heartburn non-stop from the first day here.
We arrived at the village and our team was introduced by the microfinance agent to a group of a dozen women who were her clients. After a few minutes of conversation the women gathered and sat down, with the field agent, on the ground in a large circle. Two researchers stationed themselves behind the agent while the rest of us positioned ourselves around the perimeter of the circle. I turned on the video camera and thought “Whew! We’ve been prepping this for nearly a month and now we’ll finally get to make some interesting discoveries!” But then I spent the next half hour struggling to stay focused, to listen to the conversation and watch the exchange between a woman and the field agent. Then some amount of self-awareness seeped into my head: “The breeze feels so good, gosh! I’m so exhausted, I could go to sleep right now…let me see, it’s 11ish at night in Portland…Ohh! I promised I’d call my wife today!”
Without thinking, I pulled out my cell phone and looked to see if I had a signal. To my surprise I had one bar! By walking away from the group towards a little rise I could get 2-3 bars which was good enough!
It was good to hear my wife’s voice. I closed my eyes while talking with her for about five minutes, like I was only a block away. I felt calm relief return.
But then my eyes popped open, because with the relief came a realization, triggered by my ability to connect to my wife halfway around the world while I’m in the African back country, gazing at a group of women sitting in the grass under the shade of a huge tree, with puffy white clouds against a bright blue sky. It was surreal and so powerful. I experientially understood our mission: to connect the people here to the world in a way that would make their everyday lives better, as was happening to me in the moment. Suddenly I was re-energized and fully present. Throughout the rest of the trip I kept coming back to relive this experience. It kept me energized, engaged and focused, no matter how exhausted I felt. I honestly believe it made a positive difference in what we discovered, what we surmised and in our final designs.
Tags: cell phone, focus, microfinance, presence, steve sato, uganda, war stories
Erik’s War Story: (Don’t) Go Toward The Light
By Steve Portigal at 1:11 pm, Thursday May 01 2014
Erik Moses is the Director of Research and Insights at Product Development Technologies in Lake Zurich, IL.
Not long ago I was on a project where we were tasked with understanding current practices in BioPharma labs. Overall the program was a huge success and we uncovered critical new insights for our client, which is always rewarding. But that is not what this story is about. This story is about my iPad.
As a researcher, I admit to having a bad memory. I am a dedicated note taker. I love my notes and can’t do much without them. A few months before this, I had begun using the iPad as my main source of data capture in the field, moving on from my old friend the pen and paper.
For one of our site visits we were in the Midwest at a notable university lab. We were there for the day, courtesy of our client’s long-standing relationship with this lab. That is to say, we were welcome guests. Part of the process we were observing involved a lab technician processing images in a darkroom. At one point during our visit, the PI (Principal Investigator), who was our client’s main point of contact and with whom they had the relationship, invited our group into the darkroom to understand how the process continued in this environment. Of course, I brought my iPad.
Our group piled into a cramped university darkroom and to find not only the PI, but also a few other technicians from the lab processing portions of their project. It was dark in the darkroom, so the only thing I could see was the soft red glow of dark room-specific lights.
The PI began the demonstration, while we tried not to impede the movements of everyone else in the darkroom. At some point, our participant said something very interesting that caught my attention. I thought “Hey, this is a must-have insight I need to remember!” and so I opened the cover of my iPad.
Immediately, I hear a technician behind me exclaim “Wha-what? Oh, great!” While I now recall hearing this comment just like it was yesterday, at the time I was so focused on capturing this important piece of information, I did not put it together that the technician was referring to the blunder I had just made.
After noticing a tremendously bright light in this room of black, only then did my mind stitch together the visual information of the bright light with the auditory cue of the mumbled comment. In a matter of seconds I realized what I had done.
While afterwards the PI ignored the incident and the session continued for rest of the day without another incident, I felt horrible and was flustered for some time. Reflecting on it today, I still feel flustered. I like to imagine that I didn’t mess up that technician’s experiment that much, perhaps only by hours but given what I know about that group and the process, in my heart I know I ruined at least two days’ worth of hard, time- and event-specific work.
Because of this incident I am now very careful in the field, perhaps to the point of being overcautious, often times wrapping my arms around my iPad to physically tell myself to be mindful of my actions. Learn from my experience: remember, don’t (immediately) go toward the light!
Tags: biopharma, darkroom, erik moses, fieldwork, ipad, light, note-taking, war story
War Story: Seeing Ourselves As Others May See Us
By Steve Portigal at 8:39 am, Friday April 18 2014
Here we break with tradition and present a story anonymously, to mitigate against mortification of those involved.
Twenty something and fresh out of my MA program I obtained a little consulting job which I completed from afar. The company mailed me a video camera and interview guide and sent me out to discover what people think of dinner food. I was to recruit people who would participate in a video recorded dinner we share and an after-dinner interview. I was instructed to send footage back to the company with the camera along with notes and analysis.
My first interview was with a man about my age who ate convenience foods. He was shy and awkward with me as I was with him. When I got there I set up the tripod and attempted to build rapport beyond our obvious discomfort. In an effort to focus only on him as he opened a can of soup and poured it into a casserole dish I spent very little time adjusting the equipment. He prepared soup-in-a-dish dinner and we ate together and then I went through what was left of the interview content. Perfect recruit for “convenience food eater,” and I was off.
Later at home I looked back at the video to make sure my notes are correct and to complete a partial transcript. To my surprise and immense embarrassment I realized that I set the camera up so that the composition includes only one thing in the foreground completely obscuring the participant’s head. It was a close-up view of my right breast – interrupted only occasionally by my arm each time I raised the fork. The entire dinner and interview video contained nothing more than this view. I had never met the employer or the team in person but I reluctantly packaged up the camera and my notes and sent them away without a word. Later they mention that their view of this video inspired quite a few laughs around the office. Oops.
Tags: composition, fieldwork, mistake, video, war stories
Lindsay’s War Story: Sexism in the City
By Steve Portigal at 8:11 am, Friday April 11 2014
Lindsay Moore is an independent design research and strategy consultant from Colorado.
We were in New York City, on day four of a three-week fieldwork trip. We had had some bumpy interviews the first few days, including a participant who clammed up because her husband was in the room, another who wasn’t comfortable showing us any of the software processes she had been recruited to show us, and a third with whom the conversation was like Mr. Toad’s Wild Ride, with us hanging on for dear life. But I was finally starting to settle in to the interview guide and was feeling positive about what we were learning. Plus we were getting a great apartment tour of Manhattan!
We were accompanied on each interview by a rotating member of the client team so that they could all experience the research firsthand, and this day was our first with a particular team member. Our morning interview had gone fairly well, but I could tell our client partner was having some trouble staying in the background, as she was used to more actively managing her interactions with customers.
We walked in the door for our afternoon interview, and I made some small talk, saying something like “How is your day going so far?” to our participant, who was an older gentleman. He answered that it was going much better now that we three pretty girls were there, but that it would be even better if we didn’t have clothes on. I experienced a shocked moment of “Did he really just said that???” and took a sidelong glance at my client to see her reaction. She had one of those impenetrable customer service masks of politeness on her face. I tried to shake off the comment and proceeded into the interview.
For the first 30 minutes, I found myself utterly unable to manage the flow with the participant, who would physically turn towards the client to answer my questions, and then turn back to me and say “You understand?” The interviews were about financial behavior, and he made it very clear that he thought I wouldn’t be able to follow what he was saying. Meanwhile, in an effort to be polite, engaged and responsive with her customer, my client was unintentionally making it worse. I realized I needed to gain some kind of credibility and after the umpteenth “I don’t know if you would understand” I told him that I do have some financial background and that I was following just fine. After that I was much better able to lead the interview and he engaged directly with me. Still, for another hour and a half he continued to condescend and make inappropriate/sexist comments (The number of times he suggested we “girls” go shopping at Bloomingdales after the interview? Five. What he wanted us to buy? Blouses.)
After leaving the interview I was hopping mad and said to my client and my colleague that I couldn’t believe what we had just experienced. They agreed but felt like we had still been able to uncover great information in the interview. They also thought that sometimes older men are just “like that” and that I shouldn’t let it get to me. I was bothered but decided to let it go. The interview had been uncomfortable but not unsafe, and the client was pleased with what we had learned. As an interviewer, wasn’t I supposed to be able to set my own emotions aside?
When revisiting the transcripts and coding the interview data, it really became clear to me that I was not overreacting to what we experienced. It was blatantly bad. Still, what should we have done? When I’ve related the story to other friends and colleagues, they’ve said that we should have left the home after the initial no-clothes comment. I want to agree on principle, but I also know that if I never allow myself to experience something uncomfortable, I’ll miss out on the richness and depth that is a part of this kind of work. What I do know is that it’s okay to share and talk about our own emotional responses to difficult research situations and that doing so is an important part of self-care for researchers. In the future, I will also make sure to have a plan in place with my fieldwork partners for when — and how — to end an interview, so that it’s not a process we need to invent in the moment.
Tags: fieldwork, interviewing, sexism, war stories
John’s War Story: An Ethnographic Encounter with Occupy Wall Street
By Steve Portigal at 9:00 am, Thursday August 08 2013
This story (told live at our New York book launch party) comes from John Payne of Moment Design.
“Mic Check, Mic Check!” he said with some authority as he jumped up on the planter. “Mic Check, Mic Check!” The crowd quickly repeated. “Security team needed near the information tent,” he continued. The crowd again repeated. “This is an emergency!” he added a bit more emphatically. This time when the crowd repeated they paid a bit more attention to the words they were saying. At that moment, I realized that I had just brought 25 people to one of the unstable locations in Manhattan, and none of them had signed a waiver.
In the fall of 2011, I was asked by the IxDA to give a series of workshops on ethnography and its relationship to design. By the time we had settled on an agenda and a date, the Occupy Wall Street movement had emerged, and their occupation of Zucotti Park was in full swing. It was a risky choice of research site, but what’s an ethnographer to do? We were off to Zucotti Park, or as the occupiers had renamed it, Liberty Square
Prior to our arrival in the park that morning, I had led the group through some background-key principles for participant observation, selected methods we were to try out together-and divided them into small working groups. It had been a quick preparation, but I was certain once we arrived, we’d have a once-in-a-lifetime observational experience. What I hadn’t fully considered was the possibility of imminent danger. We had just split up into teams… and that’s when the Mic Check happened.
The incident that spawned the call for security was an altercation in front of a brightly decorated tent on the north side of the park. The teepee-like structure was wrapped in a blue tarp with panels of silver heat reflective material. It was one of the more flamboyantly decorated tents, flying several flags and calling lots of attention from passersby.The occupant, an older man, had recently moved his tent partially into in a flowerbed because of the overcrowding and lack of space. As one of the few remaining open patches of ground, flowerbeds had been off limits until now.
Another man, from the south side of the park, was violently removing the teepee from the flowerbed when security arrived. Luckily, the quick intervention of the volunteer security force cooled the situation down. The incident ended shortly after it had begun with no injuries to anyone, my students included. Once I had checked that everyone was safe, my ethnographer’s instincts overcame my fear and I approached the teepee’s occupant. His name was David. He had been in the park since September 17th, day one of the occupation. I had found my respondent. He was visibly shaken, but as we spoke he let me in on a perspective that most non-occupiers would never be exposed to.
What this experience taught me is twofold: 1) Anticipate the unexpected: We weren’t sure what we would find in Zucotti Park that day, but being open to the moment gave us a glimpse of something rare-a very human perspective that stood in contrast to the stereotype that OWS had become in the media. And 2) Take advantage of your opportunities: In our case it was the last such opportunity we would get. Our visit took place on November 12th. The occupiers were evicted two days later.
For more of what John learned from his visit, check out his blog post about their time with OWS.
Raffaella’s War Story: A hot day in a bank
By Steve Portigal at 8:40 am, Wednesday June 26 2013
Raffaella Roviglioni is a UX designer at usertest/lab. In this story she experiences, like the title says, a hot day in a bank!
I like planning for fieldwork as much as carrying out those plans. But if there’s one lesson I learned from my experience it’s that no matter how well you think about any detail in advance, there’s always room for problems.
During a current project with a nationwide bank the client agreed to conduct a round of interviews with employees from four different offices located between Rome and Milan.
The day of the interviews in Milan I got an early train. I was aware of the long day in front of me: four interviews in two different offices with the lunch break to be spent moving from one office to the other.
I was fully equipped with laptop, backup recording device, spare batteries, charging chords, pens, paper, water and even some food for an emergency. I thought I covered every possible glitch or obstacle given the context. After all, I was going to a bank: can you think of a more predictable, comfortable and reliable location? I couldn’t.
It was an unusually warm day of June. The temperature was above 38¬?C (100¬?F) and after the first two interviews my coworkers and I were heading to the second location, on the look-out for a quick lunch on the go. We walked from the underground to the bank for a few blocks and when we arrived everyone was pretty flushed. All I could think about was the relief of a air-conditioned office where I could start breathing again and conduct the last two interviews.
The came the surprise of the day: the air conditioning was out of order! Meanwhile, the two employees were waiting to be interviewed so we simply sat down and started with the first one.
I had memorized the guide in order to concentrate better on the interviewee without having to look at it, but during the first fifteen minutes I had serious problems concentrating. The heat was unbearable, humidity was close to 90% and my coworkers were panting all the time. I had to exercise some yoga breathing to calm down and try to detach myself from the uncomfortable situation and be able to focus on my task. I managed to get through the interview pretty well, then we moved to the second employee’s office.
He started telling us a lot of interesting information that didn’t come out in the previous interviews but at that point we were completely burnt out. It was really hard to follow up with him. Every question that came out of my mouth seemed nothing like clever to me. Luckily for us, the employee was pretty enthusiastic about the topic and basically conducted the conversation himself, giving us a number of significant insights despite our minimal interaction.
Usually the toughest field work has to do with reluctant participants or with poor planning. In this case, it was certainly not so, but still it was very hard for me to get to the end of the day. I guess those last insights were literally hard-earned!
Tags: exhaustion, heat, hot, Raffaella Roviglioni, war stories, weather