With our Curating Consumption series Steve and I take time to look through our researcher lens at our lives as consumers. Sometimes we get to play participant and experience the other side of the research conversation. I recently participated in an online focus group for the redesign of a website. I couldn’t help but reflect on my own experience in contrast to that of being on the other side of the virtual glass. While the opportunities below were generated from a moderated online context, they also suggest possibilities for designing real time research interactions:

What Was Happening: I logged in and waited for the moderator to start talking. This was a silent discussion. Everyone was typing.

What I Was Thinking: Here I am at my computer, wearing my ear buds, ready to listen and there’s no talking. Oh my God. I’m an extrovert. How can I make it through 90 minutes of silence? How can I get my big wordy thoughts into these little text boxes? This is not what I was expecting.

Insight: No one set appropriate expectations for what ‘online focus group’ meant. I assumed it would be like a focus group with actual verbal communication. As an extrovert I found it difficult to sit quietly for 90 minutes with a virtual room full of people. As a verbal processor I struggled to articulate some of my ideas as typed words with limited character restrictions. This may have felt considerably different had I known going into it what to expect.

Opportunity: Use words wisely. If you call something X but it is different from most Xs, then clearly communicate how it is different so participants have appropriate expectations. Or don’t call it X.

Opportunity: Employ a variety of methods that cater to diverse personalities (i.e. introvert, extrovert) and learning preferences (i.e. visual, auditory, kinesthetic).  Try to avoid using only one mode of interaction, it can feel alienating and disillusioning for participants.

What Was Happening: The moderators introduced themselves and set some guidelines for the session. Our first instruction was to introduce ourselves with a story about our name.

What I Was Thinking: Fun! Our icebreaker is simple! A story about my name: where it came from, what it means, whatever I want to share! Am I allowed to talk to the other participants? I want to comment on that story!

Insight: This was a fun and simple icebreaker with a low barrier to entry (everyone has a name!). It was also appropriate to the context because all I knew of the other participants in the group were their names. We began addressing comments to each other and then the moderator encouraged us to do more of that. We quickly established a rapport and connected to each other through these stories.

Opportunity: Facilitate rapport building between researcher and participants AND among participants. If you have expectations about who should (or shouldn’t) be talking to who, clarify that at the outset (or before beginning).

What Was Happening: We were asked questions by the moderator (her type was in bold and blue) and then all of our responses would come up in a feed below. It was a small window that I was unable to resize or navigate.

What I Was Thinking: This interface is driving me nuts. I am struggling to follow all of the comments. The moderator’s questions get lost upstream when everyone starts answering. When I try to go up to revisit the question, an answer comes in and I lose my place in the thread. I cant’ find the rating scale we are supposed to use. Is 1 high or low? ARGH!

Insight: The researcher often has a clear path through the conversation in mind. Participants don’t necessarily have this big picture view and can feel lost in the forest of questions and answers.

Opportunity: Ensure participants have various tools for keeping up with the flow of the conversation. This may be easier in live/in-person meetings, and especially valuable for virtual or asynchronous interactions.

Opportunity: Provide a map of the journey that enables participants to identify where they are if they feel lost. Let them peek behind the curtain to see what’s ahead. It can be a trust builder if done well, or a spoiler alert if not framed appropriately.

If you want to play with the possibilities of using these opportunities to improve your own practice (I know I am!), you can turn each Opportunity into a question that catalyzes divergent thinking. Simply ask “How might we…” before each Opportunity (e.g., How Might We ensure participants have various tools for keeping up with the flow of the conversation?). Then challenge yourself to generate as many ideas as you can (20 is always a good round number). And if you do, please share! We would love to hear suggestions for how to improve the practice of research by improving the design of the participant experience.

Last week I had the pleasure of visiting Lisbon and presenting at UX Lx.

I gave an updated version of “Well, We’ve Done All This Research, Now What?” where we did a brief observation of the area around the venue and then developed concepts that spoke to the needs we uncovered. Among the concepts the teams played with was a giant robotic sheep that would provide shade.

The slide deck:

Per Axbom took a series sketchnotes during the session and kindly posted all of them here.

I gave a short presentation on the final day of the conference, exploring the power of user research not only to uncover data that drives product development but to change the way an organization thinks about it’s customers and itself.

The slide deck:

Sketchnotes from LiveSketching.com, Per Axbom, and Francis Rowland. Click on any of them to see the larger original.

(Side note: amusing to see the consistent use of the presenter caricature. The organizers of the conference may have contributed to this; in each attendee packet was a poster showing a funny if awkward scene with cartoon representations of all the different speakers, as well as a set of cards for one of the speakers. Attendees were supposed to trade cards until they got a complete set.)

Reed Hastings Knows He Messed Up is a Q&A with the Netflix CEO. The piece is largely snarky pseudo-hardball questions that Hastings dodges by asking us to take the long view, but this nugget about the Qwikster debacle was provocative:

Q: I’m curious if you could have done any kind of research that could have anticipated this?
A: Our focus-group work concentrated on trying to understand consumers’ perspectives on names other than Netflix.

I’m stunned by this and all it implies.

• Netflix doesn’t think of using research to understand the impact of a concept that would change its user experience?
• Netflix relies on focus groups as their strategic insight methodology?
• Netflix didn’t already have knowledge of user work flows, brand affinity, or anything else from their previous work that could have guided them?
• Netflix tested the Qwikster name and still went with it?

This is baffling, so baffling that it must not be true. There’s plenty of thought-leader UX types in house at Netflix, this is not a company that doesn’t think about design, experience, or the customer. Now where Hastings sits in that process is another question. Not to mention this quote is condensed from a longer interview and there may be some crucial content lost. But still. Stunned.

As part of the Rosenfeld Media UX Workshops Fall 2011 Tour, I’ll be leading a full-day workshop – Interviewing Users: Spinning Data into Gold.

You can choose up to 3 workshops, including ones from Lou Rosenfeld and Steve Krug. Early registration (with a decent discount) ends September 9.

Bonus: the event will held at the amazing Seattle Central Library!

I hope to see you there!

• [from steve_portigal] The Art of Design Research (and Why It Matters) [design mind] – [Lovely piece by Jon Freach on what design research brings to design and innovation.] And sometimes design teams don't have the patience to see the value in dragging out a study in an effort to make it scientifically or statistically significant. We're just not wired that way; we prefer to make and experiment and then analyze later. So what is research good for? 1. Learning about people's behavior; 2. Understanding and analyzing culture; 3. Defining context; 4. Setting focus…Design research is not "a science" and is not necessarily "scientific." It gives designers and clients a much more nuanced understanding of the people for whom they design while providing knowledge that addresses some of the most fundamental questions we face throughout the process. What is the correct product or service to design? What characteristics should it have, and is it working as intended? "The research" won't necessarily provide cold hard answers. But it will generate some good and feasible ideas.
• [from steve_portigal] CBS Radio Tells Its D.J.’s to Name Titles and Artists [NYTimes.com] – [Tying together the fortunes of radio and record sales?] Last week the head of a major radio company felt compelled to instruct its programmers to identify more of the songs played on the air, by title and artist name…at some indeterminate point in history ­ the mid-1980s ­ song identification began to vanish from the air as programmers struggled to squeeze out anything considered “clutter.” “You were always conscious about the amount of talk you would put on,” he said. “But the truth is that people tune in and tune out, and it was probably underestimated at the time how much people really wanted that information.” For record companies, having a song’s title and artist’s name mentioned on the air ­ especially if new and unfamiliar ­ is crucial marketing…“At one point in our culture there were well-schooled retailers who could help people figure out what that song was, because they wanted to buy it,” said Greg Thompson, VP at EMI Music. “In this day and age that doesn’t exist.”

I spoke at SXSW recently about the interviewing process in user research. As I’ve been working on a book about this topic the timing was great for me.

While we know, from a very young age, how to ask questions, the skill of getting the right information from users is surprisingly complex and nuanced. This session will focus on getting past the obvious shallow information into the deeper, more subtle, yet crucial, insights. If you are going to the effort to meet with users in order to improve your designs, it’s essential that you know how to get the best information and not leave insights behind. Being great in “field work” involves understanding and accepting your interviewee’s world view, and being open to what they need to tell you (in addition to what you already know you want to learn). We’ll focus on the importance of rapport-building and listening and look at techniques for both. We will review different types of questions, and why you need to have a range of question types. This session will explore other contextual research methods that can be built on top of interviewing in a seamless way. We’ll also suggest practice exercises for improving your own interviewing skills and how to engage others in your organization successfully in the interviewing experience.

Slides

Audio

Presentation (45 minutes + 15 minutes Q&A):

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

To download the audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac).

Also see

• The Rosenfeld Media page for my book in progress The Art and Craft of User Research Interviewing
• Slides and audio from last year’s presentation at SXSW

• [from steve_portigal] Oh, Etsy. How could you? [Authentical] – [Smart take on process failures in Etsy's recent misstep. User research can make a big difference] It's hard for me to believe that if Etsy had conducted user research and even informal but realistic usability testing on the idea that they would not have quickly seen the privacy violation. They could have avoided the damage control they now have to deal with because of the breach of trust they've had with buyers who already love the experience of shopping there.Etsy could have avoided the problem and discovered a possibly great idea for engaging buyers even more. Where was the business plan for allowing search of users? How does having social "circles" support the business model, exactly? How would the social media strategy be supported on the back end? More than all that, let's look at others who have gone before us: Beacon on Facebook and Boden USA come to mind. What happened there? What could the Etsy team learn from those mistakes? Oh, and, why duplicate Facebook in any way?
• [from julienorvaisas] The Art of the Police Report [Utne Reader] – [Collett provides a fascinating exploration of one cop's ability to achieve expression while governed by the formidable constraints of police report writing.] Writing is the one constant in a cop’s daily life. As with everything in the department, strict rules govern report writing, and as with any dangerous undertaking, the department will train you to do it properly. The most despised class at the police academy is the one that teaches writing. The incident report he’ll learn to write is the factual narrative account of a crime. Every event a cop responds to generates a report. Crime reports are written in the dispassionate uni-voice that’s testament to the academy’s ability to standardize writing. They feel generated rather than authored, the work of a single law enforcement consciousness rather than a specific human being. So how can I identify Martinez from a single sentence? Why do his reports make me feel pity, terror, or despair?

I was recently in Toronto to speak at OCAD (Yes, we were in this awesome building) as part of the Unfinished Business lecture series. My talk looked at the notion of culture and it’s importance for user research, and design.

Culture is everywhere we look, and (perhaps more importantly) everywhere we don’t look. It informs our work, our purchases, our usage, our expectations, our comfort, and our communications. In this presentation, Steve will explore the ways we can experience, observe, and understand diverse cultures to foster successful collaborations, usable products, and desirable experiences.

Slides

Audio

I’ve split out the presentation itself from the Q&A, which was fun, challenging, and filled with big-picture type questions.

Presentation (1 hour, including a quick intro by host Michael Dila):

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Q&A (40 minutes):

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

To download the presentation audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac). For the Q&A audio, Right-Click and Save As… (Windows) or Ctrl-Click (Mac)

Note: In the talk (and the Q&A) I refer to my interactions article, Persona Non Grata. You can find that article here.

• [from steve_portigal] A Perspective on Observation: Designer vs. Consumer [Pivot Blog] – [Nice take on my OCAD talk from earlier this week in Toronto] I liked Steve’s idea that design research is about getting down and dirty and being in the raw space to experience all the rough edges of what really happens. Because when was the last time you experienced something raw and real from a stack or research documents?

• [from steve_portigal] Armies of Expensive Lawyers, Replaced by Cheaper Software [HeraldTribune.com] – [Spin in this article is that using computers to manage super-human levels of complex data will have employment consequences.] When five television studios became entangled in a Justice Department antitrust lawsuit against CBS, the cost was immense. As part of the obscure task of “discovery” ­ providing documents relevant to a lawsuit ­ the studios examined six million documents at a cost of more than $2.2 million, much of it to pay for lawyers and paralegals who worked for months. But that was in 1978. Now, thanks to advances in artificial intelligence, “e-discovery” software can analyze documents in a fraction of the time and cost. In January, Blackstone Discovery of Palo Alto, CA., helped analyze 1.5 million documents for less than $100,000. Some programs can extract relevant concepts ­ like documents relevant to social protest in the Middle East ­ even in the absence of specific terms, and deduce patterns of behavior that would have eluded lawyers examining millions of documents.
• [from steve_portigal] PG&E launches huge paper chase for pipeline data [SF Chronicle] – [You think you have a lot of data to process? Obviously their record-keeping incompetence is just now being surfaced and they have taken on a data task that is beyond human scale. We can create systems that we can't manage!] For the past couple of days, forklifts have been carting pallets loaded with 30 boxes each into 3 warehouses outside the 70-year-old Cow Palace arena in Daly City. Friday afternoon, there were still more than 100 pallets stacked outside the warehouses waiting to go in. "There are 100,000 boxes in there, and you can't believe the papers spread everywhere," one PG&E employee said …"There are records in there going back to the 1920s. "We're looking at all kinds of parameters, and our data validation efforts are going on throughout the service area,…We're doing a 24-7 records search involving at least 300 employees and contractors, and we're working to confirm the quality of our data through collecting and validating our gas transmission pipeline records."
• [from steve_portigal] Hong Kong, 2011 [Flickr] – [My pictures from our recent trip to Hong Kong for the UXHK Conference]
• [from steve_portigal] Understanding Culture, User Research and Design with Steve Portigal – [Reserve your tickets now for either Toronto event: a lecture on March 8 and a workshop on March 9. The lecture will focus on culture, insights, and design while the workshop will be a hands-on opportunity to practice synthesizing user research data into opportunities and concepts. Hope to see you there!]

Our latest interactions column (written in collaboration with Julie Norvaisas) What to Expect When You’re Not Expecting It has just been published.

We are inevitably astounded and affected by what exists outside of explicit research project constraints.We indulged in a little reflection on some of the people we’ve met and how meeting them took us outside of the business questions at hand but had a real impact on the team and reframed the way we thought about the business questions. This opportunity to dwell on the exception provides a reminder of how these experiences deliver a potent dose of humanity to the business of providing products for people.

Get the PDF here.

Previous articles also available:

• Persona Non Grata
• Everybody’s Talkin’ At Me
• The Journey Is The Reward
• Hold Your Horses
• Living In The Overlap
• Some Different Approaches to Making Stuff
• Poets, Priests, and Politicians
• Interacting with Advertising
• Ships in the Night (Part I): Design Without Research?
• Ships in the Night (Part II): Research Without Design?
• We Are Living in a Sci-Fi World
• On Authenticity
• The Hard Work Lies Ahead (If You Want It)
• Elevator Pitch
• Kilroy Was Here
• Never Eat Anything Raw
• Content, the Once and Future King

• [from steve_portigal] User-Led Innovation Can’t Create Breakthroughs; Just Ask Apple and Ikea [Co.Design] – [This old saw again! As if user insights and strong vision are incompatible? Shame on FastCo for this hyperbolic crap.] We asked friends on the Apple design team about user-centric design. “It’s all bullshit and hot air created to sell consulting projects and give insecure managers a false sense of security. We don't waste our time asking users, we build our brand through creating great products we believe people will love." IKEA designers don’t use user studies or user insights to create their products. “We tried and it didn’t work..”Of course, neither will say this publicly since both are extremely closed companies and would risk offending users (and the design community) by speaking out against user-centeredness. Since no one will speak up, the false value of the user-as-leader has spread. The best brands are guided by a clear vision for the world, a unique set of values, and a culture that makes them truly unique and that no user insights could ever change.

• [from steve_portigal] Non-Jews Begin to Embrace Ketubah Wedding Tradition [NYTimes.com] – [Cultural appropriation of religious traditions as we continue to seek meaning through symbols] The decade of non-Jews discovering the ketubah coincides with three relevant social trends: the rise of Christian Zionism, the growth of interfaith marriage, and the mainstreaming of the New Age movement with its search for spirituality in multiple faith traditions. As a result, an increasing number of gentiles have taken up Judaic practices: holding a Passover Seder, eating kosher food and studying kabbalah, the Jewish mystical movement. “A lot of these things are grass-rootsy,” said Prof. Jenna Weissman Joselit, a historian at George Washington University, who has written extensively on Jewish popular culture. “They have to do with the growing popularity of intermarriage — openness, pluralism, cultural improvisation. And for those who are more religiously literate, they add another level of authenticity or legitimacy.”
• [from steve_portigal] More Focus Groups for ‘Spider-Man: Turn Off the Dark’ [NYTimes.com] – [What is the meaning of using consumer research? Do we admire producers for being user-centered or do we decry them for being desperate?] The producers of the Broadway musical “Spider-Man: Turn Off the Dark” are offering $60 goodie bags to people who serve in focus groups that will respond to several performances. Two focus groups attended Thursday night’s performance, and four more are scheduled to be at Friday night’s show and the Saturday matinee. It’s not unheard of for Broadway producers to use focus groups, and the musical has used them before since preview performances began on Nov. 28. But these are the first since largely negative reviews of the show by theater critics across the country were published on Tuesday. OnTrack Research, a marketing and consulting firm, is coordinating the focus groups, and here’s the rub: participants only get to see Act I or Act II, not both. They are then asked to fill out surveys and join in discussions in a “V.I.P. room.”

• [from steve_portigal] End Of An Era: Sony Stops Manufacturing Cassette Walkmans [Crunchgear] – [I share the author's surprise that this product was still being manufactured! The CD Walkman - its successor - has long been quaintly outdated, so cassettes? Perhaps there was a retro market, or perhaps other countries discarded formats differently than we have here] Sony announced it will stop manufacturing and selling these devices in Japan – after 30 years. Sony says the final lot was shipped to retailers in April this year, and once the last units are sold, there will be no cassette Walkmans from big S anymore. The first Walkman was produced in 1979. The TPS-L2, the world’s first portable (mass-produced) stereo, went on sale in Japan on July 1 that year and was later exported to the US, Europe and other places. Sony says that they managed to sell over 400 million Walkmans worldwide until March 2010, and exactly 200,020,000 of those were cassette-based models.
• [from steve_portigal] PlumWillow Is Making the Customer Part of Its Culture [NYTimes.com] – [Employment criteria: do you represent our target customer? Hiring for insight as an internship strategy] They’re part of a team of 15- and 16-year-old interns who are being tapped for their own special brand of expertise and insight: a bird’s-eye view into the life and mind of high school teenagers, exactly the audience that PlumWillow is seeking. “They definitely aren’t shy about telling us what they like and don’t like,” says Lindsay Anvik, director of marketing at PlumWillow, who helps oversee the internship program at its offices in Manhattan. The interns are also emblematic of how Web-based businesses are doing more than merely shaping their products and services around customer preferences. The companies are corralling those customers in the workplace and making them part of the design and marketing process, according to Susan Etlinger, a consultant at the Altimeter Group, which researches Web technologies and advises companies on how to use them.

The conference lineup is chosen partially based on input (i.e., voting) from the community. Even if you don’t attend, you still have a voice about what the discourse should be in our various fields. Not to mention, it’s a great way to support us! Visit each of the two talks below and click on the “thumbs up” icon. Add your thoughts, or comments as well!

Diving Deep: Best Practices For Interviewing Users

While we know, from a very young age, how to ask questions, the skill of getting the right information from users is surprisingly complex and nuanced. This session will focus on getting past the obvious shallow information into the deeper, more subtle, yet crucial, insights. If you are going to the effort to meet with users in order to improve your designs, it’s essential that you know how to get the best information and not leave insights behind. Being great in “field work” involves understanding and accepting your interviewee’s world view, and being open to what they need to tell you (in addition to what you already know you want to learn). We’ll focus on the importance of rapport-building and listening and look at techniques for both. We will review different types of questions, and why you need to have a range of question types. This session will explore other contextual research methods that can be built on top of interviewing in a seamless way. We’ll also suggest practice exercises for improving your own interviewing skills and how to engage others in your organization successfully in the interviewing experience.

For more on interviewing, you can check out our UIE Virtual Seminar and the follow-up podcast we did with Jared Spool.

Mommy, Where Do Good Products Come From? (with Gretchen Anderson)

Business case studies are the ultimate in reductionism: A complex business activity rooted in a specific context of people, company culture, time, and place is boiled down to a few key ideas. Consultants, designers, students, and people who read Malcolm Gladwell are especially prone to this form of simplification. While these simplified stories can be helpful as touchstones, we just need to remember that they are often apocryphal archetypes more than investigative summaries. Or people confuse the terms innovation and invention; looking for breakthrough ideas sends companies into a frenzied search for “new” things not great or disruptive things. In this session, we will explore some different pathways to creating great product ideas. As designers and researchers, we’re experienced enough to know that design research isn’t the only approach or even always the best approach (a point of view that Don Norman vehemently argued in recent writings). For instance, design research wouldn’t be sufficient to create a disruptive innovation like Gowalla. We’ll outline a framework that looks at different approaches to idea generation, including corporate competencies and culture, customer needs and cultural context, and technological innovation.

For more on this topic, you can check out our interactions column Some Different Approaches to Making Stuff (PDF). Also, listen to Steve and Gretchen in conversation about the speed of innovation.

Thanks for your votes!

Also see:

• From SXSW2010, slides and audio for UX Process Improved: Integrating User Insight (with Aviva Rosenstein)
• My pictures from Austin and SXSW
• My thoughts on advertising and messaging at SXSW2010