Posts tagged “recruiting”

Steve’s War Story: Details Disconnect

This story was originally published on behalf of The Field Study Handbook.

Last year I was working on a project for a financial technology client. Finding participants is often a challenge, but on this project, for small business owners, it was particularly difficult.

We had hoped to base this research on previous studies, but it proved difficult to glean details about how previous studies were done. There were rumors that another team, elsewhere in the country, had developed a segmentation algorithm, but voicemails and emails went unanswered. We heard about great participants from previous studies that we should revisit, but no one would get back to us. The schedule ticked by and the pressure mounted. In the end, we were left with no choice to work around these limitations. Finally, I began to approach recruiting agencies.

My go-to recruiting team refused to take the assignment on as they had, ironically, recruited for one of these previous studies and felt like they had tapped out the local market. Another company had gone out of business, and a third didn’t think they could accomplish the recruit.

I ended up with a recruiter I had never worked with. In the end, I think they did a good job, but a new relationship added stress to the increasingly complex recruiting process.

In our introductory call, one of our recruits expressed surprise and concern that there would two of us visiting his very small office. We eventually agreed that even though it might be cramped, it would be okay. The recruiting agency, when asked about this disconnect, reassured me that they made it clear, as per my instructions, that there would be two of us. I was confused, as the participant had insisted they had never told him anything about this.

Later that day I got an email from the participant, who sought reassurance about the purpose of the interview. He had clicked on my website (seen in my email signature) and was concerned that I was actually going to be pitching him my services. He had been involved in a focus group through this agency before, and presumed this would be something similar. I confirmed that this was not a sales pitch.

A few days later we met with him in his exceptionally cramped one-person workspace. As the interview unfolded, he abruptly stopped and directly, yet politely expressed confusion and discomfort about the interview itself. Why were we asking these questions? Who do we represent? How are we going to use this information?

It took a long, unhurried conversation about the process and our objectives to put him at ease. We resumed the interview and learned a great deal about his truly amazing businesses, past, present, and future.

I emphasize his politeness in stopping the interview, because now, when I go back to the transcript, that’s what I see. But at that time, sitting in that interview, it didn’t feel that way. It felt aggressive and angry and I spent the remainder of the interview feeling uncertain about our rapport. I overcompensated with excessive deference, people-pleasing, and probably flattery. That’s not a comfortable feeling and it’s not conducive to a good interview. I have empathy for someone feeling uncomfortable about something as odd as two strangers with a video camera coming into their office space to ask about their professional history. It’s easy to mischaracterize people that don’t “get it” as difficult. And I assume that I am pretty good at managing expectations at all the common points of failure in establishing rapport.

But boy it’d be nice if we had someone to blame. That guy was a jerkface! The recruiter didn’t do their job (and then lied and insisted they did!). Steve didn’t handle the first call or the interview kickoff properly! Yet it doesn’t seem like any of these are true.

While I felt sheepish at the end of the interview, I was surprised to get a LinkedIn request from the participant immediately afterwards. And, I guess, less surprised when I heard from him a few times weeks later about not receiving his incentive payment (This was one of the very few studies where I asked the agency to send checks after the interview was completed, rather than handing people the incentive directly myself. Mistake? I don’t know). When I followed up with the recruiter about the missing incentive, I heard in some detail how this participant had already called and yelled at the admin staff.

And so it goes.

Steve’s War Story: Giggling and Grunting

I originally posted this in 2006, and revised it slightly for Interviewing Users. I thought it was time to add it to the War Stories archive and so here’s the original version.

As we rang the doorbell, my colleague and I unconsciously straightened, preparing ourselves for that all-important first impression, that moment when our research participant would come to the door and size us up. We waited for a moment, looking at each other as we heard footsteps, mustering a smile as the inside door opened.
“Hello,” I offered, “Are you Brian?”

As I began to state the obvious, that we were here for the interview, he grunted, opened the screen door, and as we took hold, he turned around and walked back into the house. We glanced at each other, and stepped into the foyer. What did we know about Brian? Our recruiting screener told us he was 22, lived with his parents, and his brother, and was employed part-time. The rest would be up to us to discover.

It was 7:30 in the morning and we were taking our shoes off in a strange house. Eventually someone beckoned from the kitchen, and we came in. But already we were out of sync. The kitchen was small, with an L-shaped counter, and a small table for dining. The mother was at the end of the L, working with bowls and dishes and burners on the stove. The father was perched against the counter, while Brian, and his younger brother sat at the table. The father was a small man, while the other three were quite large. The room wasn’t intended for the six of us, so we managed to set ourselves up for our interview in the only place we could; at the far end of the counter, at the far end of the table. We wedged ourselves (one behind the other) on small chairs, pulling our knees in, our paraphernalia of notepads, documents, video cameras, tapes, batteries etc. clutched in close. It wasn’t ideal, but we hoped we could make it work.

But then the real challenge became clear – although Brian had agreed for us to meet and do this interview, he was actively disinterested. We were positioned 45 degrees behind him, in his blind spot. With his physical bulk, he managed to loom over his food in a way that eliminated even any peripheral eye contact; somehow this was something a smaller person couldn’t have done. His brother sat across from him, echoing his posture.

We had recruited Brian specifically, but of course, here we were with the entire family. We pressed ahead, explaining our study, and starting in with our planned questions. Since Brian was the person with whom we had the arrangement, we focused our attention on him. He would not respond, beyond one word answers (which sounded more like grunts), and the occasional glance up to his brother, causing them both to giggle.

My colleague and I avoided looking at each other (it may have not been physically possible, given the tight quarters) for fear of displaying our despair at the situation. Sure, we had arranged this interview, but the cues we were receiving were making it clear the arrangement wasn’t worth much. At this point, we had already woke up quite early to do this interview, so there was no point in giving up. If they changed their mind explicitly, they’d let us know, and we’d leave. Meanwhile, what else was there to do but press on? I asked questions, with very little response. I tried the brother, at which point Brian bolted out of the room for a few minutes, without a word. The brother was only slightly more amenable than Brian, mostly interested in making critical comments about his parents (to Brian’s great grunting enjoyment) as much as provide any actual information.

Indeed, the mother and father seemed not to have been warned that we would be coming; although I directed some of the questioning towards the mom, she reacted with pretty serious hostility, informing us (in the context of an answer to a question) that they did not welcome strangers into their house, and (while she was preparing food) highlighted the intimate nature of food preparation as a symbol, and that was even less open to strangers. The message was very clear.

But again, what could we do? Pressing on, until asked to leave, under the explicit agreement we had made, seemed the best approach. We asked our questions, following up on the information they had shared, listening closely, looking for clarification, offering up as much space as we could for them to talk, all in trying to build some flow and dialog. Even though the message was negative, at least the parents were willing to talk to us. And so, the young men faded out of the conversation, and the interview eventually switched over to the parents. Two hours later it turned out that we had completed an excellent interview with them; they each had great stories about our topic area, and revealed a lot of background about their family, about growing up, about their activities, and even their perspectives on what made the United States the country it had become.

Before we left the house, the mother insisted on cooking up some fried bread fresh and hot for us; admonishing us that “no one comes here and doesn’t get food” – reiterating the intimate nature of food she had mentioned at the beginning, but this time as a compliment rather than a shield.

As soon as we left the house, my colleague turned to me and said “I don’t know how you pulled that off; I thought we were done for and would have to leave.” I was very pleased with how the interview turned out, especially because it began at such a low point, but there was little magic to it. I didn’t try to solve the big problem of the complex dynamic we had walked into; I focused (especially at first) on just the next problem; the immediate challenge of what to say next. I was certainly keeping the larger goals in mind of how to cover all the areas we were interested in, but I was focusing my energy as an interviewer on the next thing. And by working at it in small pieces, bit by bit, the dynamic shifted. As interviewers, we had to compartmentalize the social experience of the event – the extreme discomfort and awkwardness of the early part of the interview, and just stick to our jobs. We didn’t handle the situation that differently than any other interview, and it served as a testament to our approach – listening, following up (and showing that we were listening by the way we followed up), building rapport and trust, bit by bit, until there was a great deal of openness and great information.

Years later, it’s obvious that there are better ways to communicate with the participants ahead of time to screen out unwilling participants. For example, the person who is going to be in the field should always speak live to the person they will be visiting before they day of the interview just to get that person-to-person communication started early, so both parties can get a sense of each other and start to feel comfortable (or agree that it’s not a good fit and move on to someone else). But, given the diversity of people, we will always end up interviewing people who are more or less comfortable with the process, and it’s our job to make them comfortable in order to get the information we are interested in. Doing so may make us uncomfortable ourselves, but with practice we must learn to set aside the social dynamics and focus on the question asking and listening that will make the interview a success.

Doug’s War Story: Knock-knock! Who’s there?

Doug Cooke is founder of Tinder, a research consultancy focused on people-centered innovation.

In a recent research and strategy project focused on defining a new global platform for a medical device, our research plan required us to shadow clinicians and others as they used existing devices in the “context of care.” With minor issues like HIPAA protecting patient privacy and other security issues at big urban hospitals in the US, our team decided that conducting research in Europe provided a better opportunity to understand these devices and their users.

Planning started with all the usual steps: multi-day client sessions to assess the domain, issues and problems; auditing reams of client data and documents; becoming familiar with competitive products, etc. We developed a research protocol that went through many rounds of revision with a large, multi-location client team, arriving at a clear understanding of relevant and important user issues. We developed screening criteria for participating medial institutions. Pilot studies were run at US hospitals. Months of preparation were spent in making sure our research team was fully prepared to bring back insights and perspectives that would help define the next generation global respirator platform. Ready, set, on to Europe!

Our first stop was a hospital in Wales. They had lined up the appropriate people for us to shadow and interview, including department heads, physicians, and medical techs. We spent two days shadowing, probing and gathering, and everything worked according to plan. Wahoo!

At our second stop in London (hauling two large model cases that would not fit into London’s very spacious cabs), we arrived at the check-in desk and ask to see Dr. Smith (or so we’ll call him). Upon arrival at his department wing, we learned that Dr. Smith was not in. Even more concerning was that Dr. Smith was out of the country at a conference and had not let anyone else know we were coming. After speaking with a few more people, the answer was “Please come back at another time when the doctor is in.” Ouch! In spite of all the planning, effort, and resources to get here, a few uncooperative people were about to jeopardize our research program.

How could this happen? Well, I ignored one of my primary rules: never let the client take on a critical path item that could endanger the project’s success and my firm’s reputation. Specifically, because of the difficulty of gaining access to the right people and institutions, and extremely high cost if we were to use a traditional recruiting process, our client took on the responsibility for arranging our visits to hospitals through Europe. Few clients understand the level of effort needed to screen, schedule and triple-confirm each participant. When the “research gig” is complex and requires the participation of a number of people carefully choreographed in a short time, it is essential to have a dedicated, experienced resource to make that happen.

We made it all work in the end. With no Dr. Smith and an apparent dead end, we literally started on-the-spot networking, walking up and introducing ourselves to doctor after doctor until we had made some friends that would grant us two days of access in the ICU and ER. It worked out in the end, but presented unforeseen delays and stress to an already pressure-filled project. Painful but constructive outcomes, nonetheless.

The rest of the trip in Germany and Italy presented various levels of preparedness on the part of hospitals we visited. Some hospitals were planning on hosting us for our full two day itinerary and some were expecting only a few hours meeting (which we were able to extend by turning on our best charm).

I have always been a very careful planner and can fastidiously orchestrate research logistics. I know what it takes to gather user insights. But the lessons learned from this European research foray is a clear reminders that whenever I can, I must control the recruiting and scheduling process. I hope to never again knock on any unsuspecting doors.

Gerry’s War Story: Right to be Wrong

Gerry Gaffney runs the UX consultancy Information & Design in Melbourne. He publishes the User Experience podcast and is current director of publications at UXPA.

I was researching, with my colleague Patrizia Bordignon, how people thought about and dealt with home renovations.

One of the methods was a diary study (“cultural probe”), and we had carefully recruited – or so we believed – a small set of participants with whom we would work for several weeks.

Warning bells sounded fairly early with one of the participants, who showed up very late for the initial briefing. These things can happen, so we ran a separate briefing session for him, gave him his kit of reporting materials (camera, diary and so on) and sent him on his way. Let’s call him Mr. W.

Three days after the briefing we telephoned each of the participants. It’s a good idea to do this to remind people about their commitment, to redirect as necessary, and to address any issues that arise. All our participants were on-track, with the notable exception of Mr. W, who seemed somewhat evasive in his answers.

At the end of the first week, we visited the participants. Again, this is good practice; it’s an opportunity to see how the data is being gathered, and what changes might be needed to the process. We also use that opportunity to make a part-payment to the participants, which can serve as a nice motivation.

We were delighted with what we saw. Participants had kept bills and receipts, photographs and magazine clippings, they showed us their renovations or their plans, and we were confident that we were getting plenty of highly relevant data.

When we visited Mr. W’s house, however, it was evident from the first moment that his home was different. The front gate didn’t work properly and the hinges squeaked, the garden was unkempt and the house gave an overall sense of dilapidation. Inside it was a similar story. Every room was in dire need of immediate restorative work, but none was evident. I felt a tad depressed as we drank tea from cracked mugs and listened to Mr. W list the things that needed to be fixed.

Mr. W was not an enthusiastic renovator. His house represented a series of urgent and necessary tasks, none of which had been tackled.

It looked like we would collect no useful data from Mr. W, and as we traveled back to the office we talked about our disappointment and reexamined our recruiting strategy.

However, as we moved into data analysis, we found ourselves referring quite often to Mr. W, and gradually came to realize (no doubt this should have been obvious earlier) that Mr. W’s world was in fact directly relevant to our project. While the enthusiastic renovator was undoubtedly a key consideration, the unenthused or reluctant could also present great opportunities. Their needs and goals were different, their attitudes were different, and the way that we would design for those characteristics was different.

In many ways, in fact, Mr. W was an ideal participant specifically because he didn’t fit our expectations. He challenged the underpinnings of the project, and made us examine our design decisions in a much more rigorous fashion.

I often reflect back on this experience when I’m doing user research, and I specifically watch out for negative reactions and experiences, because they can often teach us things that we might not otherwise learn.

I still believe it’s important to recruit carefully, but perhaps we should be more open to the idea that the “wrong” participant is sometimes precisely the right one.

One way to recruit in-home research participants

Credit card issuer Capital One isn’t shy about getting into customers’ faces. The company recently sent a contract update to cardholders that makes clear it can drop by any time it pleases. The update specifies that “we may contact you in any manner we choose” and that such contacts can include calls, emails, texts, faxes or a “personal visit.”

Yep, that’s how you get willing research participants – add it to the Terms of Service! Sure, I’m kidding; that’s not really what this article is about (it’s about the credit-card company claiming rights to repossession for non-payment). Still, this ups the creepy ante for visiting customers and makes the trust aspect of recruiting participants just that much harder.

Full story

Cordy’s War Story: A Crisis of Credibility

Cordy Swope (Twitter) describes himself as either a Design Researcher with Grey Hair or a design researcher with grey hair.

IDEO. NYC. Early 2010.

I had been summoned from Europe to lead a project about the future of education in the US. At IDEO, there is a well-established a code of ethics for site visits. This code takes extra measures to protect the privacy of informants – especially their identities and contact data. IDEO also has sensible, street-smart guidelines for fieldwork in sketchy environments. In previous jobs, I had seen a situation in which two of my female design researchers had to go to remote, sparsely populated parts of the Midwest and visit big, burly, smiling men who stored every conceivable power tool in their dungeon-like tornado cellars.

There is never a shortage of people in NYC though, and recruiting there offers many delights. For instance, NYC is one of only several places where it is possible to recruit for impossibly specific profiles like: “Seeking 3 single dads who have volunteered with their children at a local charity organization within the past 2 weeks, and who also must struggle with their own gender identity and make at least $150K/year.” In the Tristate, if you are one in a million, by definition there are at least 22 of you.

Our recruiter used Craigslist for most projects and straightaway found us one of our targets: a working mother who had successfully completed a BA online while still raising a family. I had a new team and my associate design researcher was an eager, empathic and articulate ethnographer doing her first project at IDEO. We headed out to Inwood in Brooklyn for our first site visit, hoping to get insights from this working, baccalaureate mom.

During the ride, I played the senior mentor guy, offering advice about doing ethnography “in a design context.” We arrived at the address in Inwood, an obscure part of Brooklyn that looks like a sad, dilapidated part of Queens that in turn, tries to look like a nondescript suburb in Long Island. We were buzzed into the building, walked up to a door and were greeted by a large woman with a curly red mane of hair. Her name was “Roberta-but-call-me-Bert.”

She let us in. The apartment was dim. It smelled of litter box mixed with burnt Dinty Moore beef stew that Ramon, Roberta-call-me-Bert’s husband had overheated on the stove. The dingy plaster walls were covered in old shopping lists, written in a mangled scrawl that suggested vaguely menacing pathologies and personality disorders suffered by their author.

The sofa we sat on smelled of cat piss, and the living room offered up no pretense of ever having been cleaned. We sat up straight, made eye contact in that standard, pious, non-judgmental manner that earnest ethnographers often adopt. We began the paperwork. We were offered water and politely declined.

I asked her about work, family, free time; all of the perfunctory questions before we got into her BA experience. Since I was the seasoned professional, I led the discussion, “Tell me a story about your favorite class…”, “Did you make friends with your classmates?”, “Do you still keep in touch?” Since my associate was taking notes, I focused on keeping the discussion moving and letting Roberta-call-me-Bert lead us to all sorts of exciting insights.

The trouble was, she didn’t.

“Oh, I don’t remember much about that class,” she said about her favorite statistics course she took just before graduating 18 months ago. “Yeah, I pretty much kept to myself, because I had to work and raise a family, you know?” I nodded my head earnestly.

I began asking her questions about change: “Do you view your daughter’s education differently now since you got the degree?” “Not really,” she said, as her daughter ate ice cream from a container while watching a YouTube video about dog fighting.

We eventually went on our way. Once out the door, I was about to launch into the debrief. Since I was the experienced one, I was going to teach my associate a simple, time-honored 20 minute structure I often use for debriefs: Interesting Behaviors/Motivations and Drivers/Problems and Frustrations/Opportunities.

I noticed that she was grimacing.

“What’s wrong?” I asked.

“That was a waste.” she replied.

“What do you mean?” I asked.

“She lied, she never went to college.”

I was gobsmacked.

And she was absolutely right.

There were no interesting behaviors. There were no drivers or motivations. There were no problems or frustrations. There were no opportunities.

There was no diploma. It was “packed away somewhere.”

We returned to the office. Another colleague was leading a project in men’s fashion and desperately trying to recruit shop-along dyads of couples in their 40’s and 50’s where wives selected the husband’s clothes. She said they had already recruited one couple on Craigslist and that her name was Roberta from Inwood, Brooklyn.

Sean’s War Story: Pockets full of cash

Sean Ryan, a corporate ethnographer, reflects on a fieldwork experience where he learned first-hand some crucial lessons when going into another country: pre-recruit participants, and do some basic homework about where you are going.

It was back in my early days as an ethnographer. I was still a young pup in the field, doing consulting projects. I was teamed up with an Elder Anthropologist – a Puerto Rican woman who lived in Guadalajara, named Luz. We were doing a project for a major pharma company who had just had great success with a new oral care product, so they thought they would try an ethnographic exploration to uncover any other unmet needs. I think their aspirations at the time were something like “We want the next $500 million consumer product!” Luz and I were to visit two field sites in Mexico: Guadalajara and Tijuana. Living in Los Angeles, I was relatively close to the border and it wasn’t yet seen as that dangerous to go to Tijuana (i.e., no Mexican mafia drug lord street battles…at least you didn’t read about them in the papers everyday). But I still had my reservations, not possessing any potent Spanish language skills (outside of the slang I had picked up from bartending in a Mexican restaurant in Long Beach).

Having only been to Tijuana once to explore the finer points of Avenida Revolución (read: drinking tequila shots with college kids and having my head shaken back and forth by a woman with a whistle), I had no real frame of reference for doing fieldwork in TJ. As I quickly learned, neither did my counterpart Luz. She had some relatives in TJ, but had never done fieldwork there. And so we made what we later realized was a critical error in not pre-recruiting participants before we went into the field. Upon arriving in Tijuana we quickly found ourselves literally approaching people in the streets, in shops, etc. to ask about their oral care routines (a strange encounter for the locals I’m sure). While this has all the hallmarks of classic guerrilla recruiting it’s never a comfortable situation to be in, especially in a foreign country. Luz was doing her best to recruit people while I stood by idly awaiting our field day fate.

Eventually we started to have some success…or so we thought. One woman who worked in a nice department store in downtown TJ offered to let us come to her home after work. We got her contact info and told her we would see her that evening. We were offering $150 in US cash (this was more than 10 years ago) to interview the participant and observe their oral care routines. This, no doubt, was more than substantial for an incentive. So we were quite confident that we would have no problems grabbing participants on the fly. That evening, we made our way to this woman’s house via an old Crown Victoria station wagon taxi (with the suicide seats facing out in the back). Once we got to her town we approached the participant’s door and gave it a confident knock…but nobody answered. We waited a few minutes longer and knocked again…still no answer. This was before mobile phones, so we couldn’t exactly call this woman on her cell. We sat and waited for 15 minutes, but then realized that our day was quickly wasted on a participant who, for whatever reason, decided she did not want to do the study (Perhaps she thought the $150 was too good to be true?). In a moment of desperation, Luz decided to frantically go door-to-door in this small community, hoping for a shot at someone’s teeth and mouth. But to no avail.

This disastrous field trip continued. The next day we tempted fate again by preying on another unsuspecting citizen of Ciudad Tijuana. Once again, we arranged to go visit a shopkeeper’s home later in the evening. Once again, we had no idea where exactly our little field visit would take us. And once again we crammed ourselves into an old Crown Victoria station wagon. This time we were left off at what appeared to be a small village of Gypsies. It was, in fact, just a typical working class abode on the outskirts of the city. I brazenly brought out my Sony DV camera with the Carl Zeiss lens and began filming the local scene as we walked through the streets to find the right home. We were very excited to actually find the participant’s home and then to actually find her in her home!

It was a very interesting interview: the participant was a mother of two, a 9-year-old daughter and a 6-year-old son. We observed their oral care routines, which consisted of going out to the backyard to gather water from a large plastic drum (as there was no running water), after which the children vigorously brushed their teeth with your standard run-of-the-mill Colgate toothpaste and toothbrush. When we paid the mother $150 (US) cash at the end of this encounter, her eyes lit up. I realized at that moment that this was probably more money than they made in a month. And so we broke another field rule: understand your surroundings and pay participants appropriately based on the context. But there was bit of a feel-good moment here too; the client could clearly afford the incentive money, so it was no skin off of their backs.

After this first round of field visits in Tijuana we came back about a month later for a second round, with different participants. We interviewed a relative of Luz’s who lived in a canyon high above where we had visited last time. He laughed out loud when we told him that we had been down in that village only a month ago. He said with all seriousness “Don’t you know that is the most dangerous area in all of Tijuana?!” Of course we had not known this. I thought back to the $800 Sony camera that I slung around in the streets of that village. And then I thought of my pockets full of cold, hard US dollars. I laughed to myself, but thought “I need to be a little more careful in the future if I’m going to make a career of this ethnography business!”

Leo’s War Story: No, We Really Meant the User

Product Design Manager Leo Frishberg underscores the effort required to ensure you’re seeing the right user in the right context.

Our team was embarking on an ambitious, multi-country Contextual Inquiry effort. We had created our sample cells, identified the right industries, established a great relationship with our sales team and done All The Right Things Up Front to make the effort a success.

Working from Oregon with prospective participants in Bangalore is never an easy prospect; introducing a new research technique at the same time raised the stakes.

Several weeks in advance of the interviews we had contacted our sales team in-country explaining the process: we needed individuals who were currently working with our equipment and willing let us observe them working in their labs, in situ.

Everyone claimed to understand. We arrived in-country and I confirmed the arrangements, on the telephone, with the sales team. “Yes,” they confirmed, “we’ve found exactly who you are looking for…”

We arrived at our first interview in a gorgeous sparkling new office building and were led to an upstairs glass-enclosed conference room. Presently, a manager-type entered, clearly expecting to hold court with us.

I began the discussion with a recap of our expectations and a quick sanity check with the individual.

“So,” I began, “we are looking forward to working with an actual user in the lab. Are you going to work with us today?”

“No,” he said, dismissively. “I’m the team manager. I can tell you everything that’s wrong with your equipment. I’ve polled the team and have collected answers from all of them.”

It’s at times like this, having flown 10,000 miles, having spent as much time as I had setting things up, that I lose a part of my conscious brain. I could feel the anger rising, but I knew that couldn’t help improve the situation.

Instead, I signaled to the sales guy sitting next to me that as far as I was concerned, the interview was over and we could pack up to go to our next meeting. Here’s where the details get sketchy, but I know he said something, in English, to the manager, and whatever magic words he uttered, the manager smiled and nodded, suggesting he could definitely get the lead engineer to help us. He left to find the guy.

A few minutes later, the engineer entered the room, curious as to what the group was doing there. We began the front part of the interview, and it was clear he was the right guy. After explaining what we were planning to do, we asked if he had any questions or needed any further explanation.

‘No,” he said, “you want to see me work with the equipment. I don’t have anything to do today, but I could show you what I was doing last week.”

That was fine, we agreed.

“Okay. Just give me a few minutes and I’ll bring you back…”

Imagining what he might be doing in those few minutes I stopped him. “Uhhh, what would you be doing between now and then?”

“Oh,” he assured us, “I’m just going to get the equipment all set up.”

“Great!” We practically shouted. “That would be great! We’d be happy to watch you do that!”

He smiled as if hoping we had taken our medication and led us to his lab. “I’m not sure what you’ll find so interesting about my pulling the machines off the shelf, but come on along…”

The take-aways remain the same:

  • Persistence and staying on track no matter what the situation throws you
  • No matter how much you prepare, nothing will go as planned

This Week @ Portigal

Monday is well underway and the week is filling up with meetings and work sessions! Away we go!

  • Last week we kicked off a super-rapid project. We didn’t know we were doing the project at the beginning of the week and by the end of the week we had started recruiting research participants. This week we’re lining up our participants and figuring out what we’ll do in the field.
  • I’m calling it “collaborative listening” – thanks to our officemate Olly, we’re experimenting with some networked speakers that lets us all listen to music together instead of individually over headphones. This will mean sorting out some social norms around volume, phone calls, and musical tastes. But so far, so good (oh yeah, because we’re listening to my music right now!)…
  • We’re hosting our first event later this week. We’ve invited a small number of folks for a discussion and will be sharing more once it’s all over. But we’re actively discussing our catering options right now!
  • More conference submissions to prepare, more conference acceptances to announce, and more conference presentations to start getting together!
  • This week we’ve begun reaching out to potential new teammates, partners, and collaborators. We don’t know where we’ll end up but the journey is sure to be an informative one.
  • What we’re consuming: A Visit From The Goon Squad, The Firestarter Sessions, Pizzeria Delfina

ChittahChattah Quickies

Why You’re Doing Customer Research All Wrong [Inc.] – It comes as no surprise that many innovative ideas hit the cutting room floor before ever seeing the light of day in consumer testing. The author suggests that too many great ideas don’t get chosen for testing and this is where the problem lies. While I agree that this is a grave problem for customer research, it’s not nearly as reprehensible as the omission of consumers from ideation sessions, and the failure to converge in the ideation process. In fact, I’d argue that the problem could be averted with two steps upstream in this process. First, start with the end in mind when planning a brainstorming session and invite customers and executives to help generate stakeholder-inspired ideas. Secondly, make sure those ideas get clustered and prioritized before anyone leaves the room. Ideation should include both divergent and convergent thinking! This results in more collaborative value-added ideas and less ‘intuitive’ choices about which ones merit further testing.

Affinnova studied 100 testing campaigns that its clients had done in the past. Typically the testing process went like this: A company came up with a long list of potential ideas to test, whittled it down using mostly executives’ intuition, and then tested the much shorter list of ideas. Affinnova, on the other hand, took the initial brainstorming list and tested everything on it, presenting the ideas in groups and asking participants to select their favorites.

Looking To Hire And Keep Great Innovators? Focus On The 3 Rs [Co.Design] – When companies look inward in a quest for amping up their innovation capabilities, they undoubtedly see the potential of their human resources. The three Rs of getting and keeping innovative employees are Recruiting, Retraining and Rewarding. Given the very premise of the article a fourth R, Reflection, seems mighty important. While the ROI (yikes, another R word!) of a strategic debrief may be hard to justify in some cases, the cost of ignoring valuable lessons learned from experience can be catastrophic. Consider how many times companies learn the same lessons over and over again. It’s Ridiculous. Besides, a healthy organization that engages its employees in regular reflection is likely to keep those folks feeling engaged, valued and loyal, thereby reducing the need to look outside for more innovators.

Innovation relies on people more than other processes. This reliance on employees, management, and executives in an organization requires that the “right” people are attracted, and then given the appropriate tools and techniques for a sustained innovation success. Their passions and capabilities also must be ensured to align with the needs and expectations of the firm.

Building Self-Control, the American Way [New York Times] – Although this article is focused on parenting strategies for cultivating self-discipline, I think the lessons can be applied to nurturing innovative thinkers. This article talks about the importance of play in allowing children to practice and develop skills like self-control, self-esteem and social interaction. Companies who rely on their people to continually generate creative ideas should explore opportunities for productive play experiences that challenge and nurture their employees’ essential abilities to manage themselves through intrinsic motivation.

Fortunately for American parents, psychologists find that children can learn self-control without externally imposed pressure. Behavior is powerfully shaped not only by parents or teachers but also by children themselves. The key is to harness the child’s own drives for play, social interaction and other rewards. Enjoyable activities elicit dopamine release to enhance learning, while reducing the secretion of stress hormones, which can impede learning and increase anxiety, sometimes for years.

 

Julie’s presentation on “Finding the Right People”

I recently had the privilege of guest-lecturing for an undergraduate class at the Center for Design Research at the University of Kansas taught by Julia Eschman and Tamara Christensen. The topic was the importance of recruiting the right people as participants to drive powerful insights for design research. I addressed the strategic reasons why, as well as tactics for doing so.

Recruiting is a key component of the design research process. It is to the detriment of project outcomes when it doesn’t get the attention it deserves. I know, I know… it’s a lot more fun to talk about sexy new methodologies and while away our days ideating, but before any of this can happen teams need to think hard about the source of the inspiring data and stories: people.

The presentation is below:

This was the first time I’ve presented this material, so take a look and please share some of your key experiences, snafus, complaints, challenges and learnings and etc. as you find the right people for your projects.

ChittahChattah Quickies

  • [from Dan_Soltzberg] Power Players and Profanity: Talking About Talking Dirty on NPR [Bob Sutton: Work Matters] – The author of "The No Asshole Rule" talks about the role of profanity in work culture and leadership, and his recent interview on the topic for NPR All Things Considered
  • [from julienorvaisas] How facts backfire [The Boston Globe] – Researchers at the University of Michigan found that when misinformed people were exposed to corrected facts in news stories, they rarely changed their minds. In fact, they often became even more strongly set in their beliefs. Facts, they found, were not curing misinformation. Like an underpowered antibiotic, facts could actually make misinformation even stronger. Instead of changing their minds to reflect the correct information, they can entrench themselves even deeper. The general idea is that it’s absolutely threatening to admit you’re wrong,” says political scientist Brendan Nyhan, the lead researcher on the Michigan study.
  • [from julienorvaisas] Eerie relic of science history [Boing Boing] – [It's worth reflecting on the ethical boundaries of recruiting every once in awhile – though the Milgram study's ethical lapses went well beyond the use of questionable recruiting methods, of course.] This is the newspaper ad that recruited subjects for Stanley Milgram's obedience to authority experiments. As you can see, subjects were told they were being recruited to aid research on memory and learning. In reality, Milgram was studying how far down the path of evil average people would go, simply because someone in a lab coat told them to.

Reading Ahead: Managing recruiting

Reading ahead logo with space above

There’s always something new in every project. Often we encounter a bit of process that we may not know how to best manage it. So we’ll make our best plan and see what happens. We learn as we go and ultimately have a better way for dealing with it next time.

In a regular client project, we write a screener and work with a recruiting company who finds potential research participants, screens them, and schedules them. Every day they email us an updated spreadsheet (or as they call it “grid”) with responses to screener questions, scheduled times, locations, and contact info. It still ends up requiring a significant amount of project management effort on our end, because questions will arise, schedules will shift, people will cancel, client travel must be arranged, etc. etc.

For Reading Ahead, we did all of the recruiting ourselves. Although we’ve done this before, this may be the first time since the rise of social media: we put the word out on LinkedIn, Twitter, Facebook, email to friends, and here on All This ChittahChattah.

While Dan lead the effort, we both used our own networks, and so we got responses in a number of channels, sent to either or both of us, including:

  • @ replies on Twitter
  • direct messages on Twitter
  • Comments on Facebook posts
  • Messages on Facebook
  • Emails (directly to either of us, or forwarded from friends, and friends-of-friends

facebook
A private dialog on Facebook

facebook2
Comments on a Facebook status update. Note that Dan is able to jump in and make contact directly

twitter
Direct Messages in Twitter

Some people were potential participants, some were referrers to other potential participants, and some were both. And given the range of platforms we were using, with their associated restrictions (and unclear social protocols), we had to scramble to figure out who could and should communicate with who to follow up and get to the point where we could see if the people in question were right for the study. We didn’t expect this to happen, and eventually Dan’s inbox and/or his Word document were no longer efficient, and as some participants were scheduled or in negotiation to be scheduled, he ended up with this schedule cum worksheet:

schedule

Being split across the two of us and these different media, eventually we were interacting with people for whom we had to check our notes to trace back how we had connected to them, which was great for our sample, since it meant we weren’t seeing a group of people we already knew.

It was further complicated when we had finished our fieldwork and wanted to go back to everyone who offered help close the loop with them, thanking them for help. Technically, and protocol-wise, it took some work (who are the people we need to follow up with? Who follows up with them? What media do they use), basically going through each instance one-by one.

twitter2

We haven’t figured out what we’ll do next time; we won’t forget the challenges we’ve had but there’s just not time or need right now to plan for the future. If I had to guess, I’d imagine a Google Spreadsheet that includes where we got people from, who owns the contact, whether they are participant-candidate or referrer, etc.). Despite being very pessimistic about the demands of recruiting, we still underestimated the time and complexity required for this project.

Reading Ahead: Figuring out who to talk to

Reading ahead logo with space above

People always ask us, “how do you find the people for your projects?”

Figuring out how to identify appropriate people to interview for a project is all-important. For Reading Ahead, we know we need people who are active readers. What constitutes an “active reader?” We’re defining it as people who read books at least three times a week, in multiple locations. We want people who are engaged in the behavior at a level where they will have lots of experiences from which to draw. We also know that we want to look at how people’s behavior changes/doesn’t change/is supported by/is influenced when reading books in print vs. reading eBooks using a device.

When we have established the criteria for participating in the research, we typically use a specialized recruiting company to find people. We write a screener, which has a series of specific questions to identify people who meet our criteria.

screener
Screener excerpt, Reading Ahead project, 2009

Finding the right people can be quite complex, and for some projects, we’ve written screeners that are more than 10 pages long. If we’re looking for people who do activities X and Y, in locations 1, 2, and 3, but have never done activity Z-well, you get the idea!

In this project, the criteria are simpler, and we’ll be doing our own recruiting. In fact, if you’re in the Bay Area and an avid reader or Kindle user, let us know and maybe we can talk with you!

Update: We put together a representative screener that is formal enough to be given to a recruiting firm, even though we aren’t doing that for Reading Ahead. You can download it here.

Series

About Steve