Interviewing Users is now available. Get your copy here!







Excerpting from a great post…The Art of Observation and How to Master the Crucial Difference Between Observation and Intuition [Brain Pickings] – Highlights from a 1957 book by Cambridge University professor W. I. B. Beveridge come from the era of the scientific method but are broadly applicable to creative, innovative, design-thinking approaches to problem solving.

Ultimately, Beveridge argues that the art of observation depends on developing the capacity for pattern-recognition, which in turn relies on a broad pool of networked knowledge that allows you to spot the piece that doesn’t fit: “In carrying out any observation you look deliberately for each characteristic you know may be there, for any unusual feature, and especially for any suggestive associations or relationships among the things you see, or between them and what you know. … Most of the relationships observed are due to chance and have no significance, but occasionally one will lead to a fruitful idea.”

Interviewing Users is now available. Get your copy here!

Here’s a really interesting project about How People Talk to Themselves in Their Heads.

He would ask them to wear a microphone headset attached to a digital recorder and speak aloud their thoughts as he followed closely behind with a camera. He would not be able to hear what they were saying…[The] videos are simultaneously naturalistic and as objective as possible. In the lab, in front of a researcher, people are often reluctant to reveal exactly what they are thinking. Writing a diary of inner speech is somewhat more private, but many people find it annoying to regularly drop everything and make an entry; sometimes it’s difficult to remember what one was thinking about even minutes earlier. In Irving’s videos people are living their lives more or less as usual, walking and talking to themselves as though they were unaccompanied. Of course, people who are not completely comfortable with the scenario sometimes speak into the microphone as though trying to entertain someone else. And getting people’s inner speech on tape captures only linguistic forms of thought, neglecting the kind of thinking that happens in images and scenes, for example.

The notion that unfettered, deeper self-exploration and self-expression can come when not interacting with the interrogator – to the point here of nearly eliminating the interrogation entirely – evokes the (I presume mostly obsolete) approach to therapy where the patient does not face the therapist.

I find the videos compelling (and voyeuristic to the extreme). Check out the other videos at the link – the ones that take place without the strolling seem more like a diary and less like a peek into the stream of consciousness. But for each of them, see if they pass the sniff-test for you: is the person talking the way they are because of the experiment (they know they are being recorded; they feel they need to come up with something to say, they are aware of their own “voice”, etc.) or is it really getting as deep as the researcher claims? I was mostly convinced but a sliver of doubt remains.

In the research we do, we make no claim of naturalism; we certainly want to direct and influence what is being shared, and we build rapport to facilitate openness and honesty. This approach isn’t likely to be appropriate for us, but it’s certainly provocative to look at the output of an opposite approach – where the interviewer is effectively absented and rapport is not a consideration.

Tom Wood is one of the partners at Foolproof, an experience design firm based in the UK.

About 10 years ago I was trying to understand online poker playing behaviours on behalf of a gaming company. We’d recruited for a study across their various target segments, but the hardest to find were the high-value, semi-professional players. They prized their anonymity and guarded their playing secrets.

One of the respondents I did find was a part-time property developer, part-time drummer, but his passion was poker. He was close with players from the city’s professional soccer team who were happy to lose large amounts of money in order to pick up skills in poker: an important accomplishment for the professional sportsman in the UK.

The interview did not go well at first. The respondent was a regular online player but his behaviour when using the subject site was stilted and he seemed so disengaged that I began to worry that he was out of his depth online. Eventually I decided to reframe and go back the beginning of the discussion, where we had talked about his usage habits on his regular site. This time, because he was getting more relaxed in my company, I suggested doing this by watching him play. The key behaviour this revealed was how he found a table he wanted to join. This involved simultaneously watching a large number of games in progress – an almost incredible skill. What he was studying was the weaknesses of the players at the various tables: their inexperience, bravado, impatience, petulance. His whole demeanour changed, and I had a feeling like being a naturalist watching a lion selecting the impala that it is going to turn into lunch. Compelling but horrifying at the same time. It was clear that the subject site I’d asked him to use had poor affordance for this important process, but because it was a basically unchivalrous activity he had been guarded about discussing it.
This change in tack got me this and other insights which informed our design advice. And resolved me never to take up poker.

Most experience design folk enter the field because they understand that they themselves don’t have all the answers. I’m fond of this story because it was when I properly realised that I didn’t even have all the questions. I suspect that this job made me a better researcher, and certainly made me approach certain types of work in a completely different way. At Foolproof we always preface our discussion guides with words to the effect that the discussion guide is just that, a guide – and that we reserve the right to take any approach we need to in order to meet the research objectives.

Ilona Posner is a User Experience and Usability consultant with more than 25 years of experience. In this story, she is challenged in different ways to leave her participants in relatively good shape.

Around the year 2000, homes with internet service were rare. AOL was plastering the planet with CDs that promised free internet. Modems were uncommon and expensive. Online access usually required a modem card installed inside a computer case by a service technician, at a significant cost. My client, the largest Internet Service Provider in Canada, was redesigning their Self-Installation Package for its DSL service; today this would be called a DIY kit.

The goal of our research project was to evaluate the customer experience. It entailed contacting customers who had just ordered the package, interviewing them about their order experience, and asking to visit their homes to observe the installation of the hardware and setting up the service. We visited many homes and observed people with diverse technical experience trying to install this package. The success rate of the customers completing this self-installation within our allotted 2 hours was very low. We had to suffer silently watching their ordeals: searching among numerous papers and user manuals that accompanied the package for the correct documents and locating the required identification codes; mixing up phone and internet cables; moving their furniture so that the provided cables would reach their destinations; and trying to explain their problems in repeated phone calls with technical support. In some cases, after observing them struggle for 2 hours and realizing they were incapable of completing this task unaided, we felt so sorry for them that before departing we completed the installation process on their behalf. We felt bad that they would have to spend additional days waiting, making additional phone calls to arrange for a technician’s visit, and dealing with the additional costs of assisted installation. That way, we also were able to witness their excitement and gratification of getting online; for some it was their first time.

I clearly remember one participant who actually was able to successfully complete the installation, and it “only” took him 1.5 hours to do it. He was a male in his early 30s, technical writer by profession. His PC had 32 MB of RAM, and was running Windows 95. He already had a modem but was switching to this new High Speed Service. He had to remove the internal ISA modem card from his PC tower in order to install the provided Ethernet card. He was more confident and comfortable at this task than most of our other participants. While our camera rolled, he confidently skimmed documents and manuals, even when they were different manuals from the devices he was dealing with at the time. He opened his PC without difficulty. He proceeded to remove the internal modem card from deep inside his PC case. In the process, he cut his hand on one of the sharp internal edges of the metal case. His hand started to bleed! Blood got on his hardware. We had to interrupt our observations to assist him in stopping the bleeding.

After completing our research, we redesigned the package. We reduced the number of documents and numbered each one for easy reference (unfortunately, this simple and usable solution only lasted until the next rebranding exercise conducted by the marketing department, who did not inherit our design rationale). We rewrote the instructions, using beautiful visuals. We also included a special highlighted warning, “Please be careful when opening your computer case, there are many sharp edges inside.”

I wonder if anyone ever noticed that warning message.

I was in my old hometown of Toronto last week for Interaction13. Of course, I did spend some time wandering and (shivering and) taking pictures. The Flickr set is taking shape here but meanwhile some faves for you are below.

Sharon Cartwright, a consultant with Optimal Usability, shares her story about uncovering context.

We were surprised that Anna wasn’t at home when we arrived at her house. We could see through the lounge windows that the house was empty of furniture and personal belongings, adding further to the intrigue. We had followed our usual protocol of sending an email confirmation clearly stating the time of the research session and Anna had been called the day before to confirm.

We decided to try calling Anna. She picked up our call as she was coming up her driveway. She was running late and seemed a bit agitated, telling us that she had locked herself out an hour earlier. She was just moving into this house and didn’t have a spare key yet. She’d decided, in the interest of proceeding, that she was going to smash a window to get in and had just called around to her brother’s house to pick up the appropriate tools.

Anna seemed like a practical woman, and smashing her front door window didn’t seem to daunt her. I indicated that there was no need to take such drastic action for our sakes. But she was adamant and was soon taking a hammer to the glass panel in the kitchen door.

Once inside, Anna, myself and the client (who’d come along for the ride) began the clean-up. I manned the vacuum cleaner and mused over the start to the session. There is something bonding about a shared clean-up!

We were there to observe Anna set up wireless broadband. On a good day the process wasn’t straightforward; we had already seen several participants struggle through it. Over the next two hours we observed Anna encountering several technical issues with hardware and software. She managed to resolve some issues on her own. Many times she resorted to calling the contact centre, although their advice was mixed – sometimes helping Anna, sometimes complicating matters.

We had scheduled the session for 1.5 hours expecting this to be sufficient, but many sessions – including Anna’s – extended over this. After two hours Anna had not succeeded in setting up her wireless broadband connection. Unable to stay any longer, we were disappointed to leave without seeing Anna ultimately succeed. We wished her luck, as she was clearly going to need it.

During the two hours Anna revealed a few things to us. Her long-term relationship had recently ended, explaining her move to Auckland. She was also looking for work. Her ex-partner’s teenage son had generally taken the lead on the technology front in their home, and it was dawning on her that this was now her role.

With the significant life stresses she was facing it was hard to watch her struggle through a technology set-up that should have been easier, dare I say simple. While we learned that setting up wireless internet often happens during a time of stress, as it’s one task of many when you move house, I felt for Anna and the difficult time she was going through.

Last week I was in Ottawa speaking at Carleton University as well as delivering the closing keynote for UXcamp Ottawa. Here are some photographic highlights of my time there. Look for the rest of my pictures on Flickr.

Joseph Henri Maurice “The Rocket” Richard

Suzy Q Donuts.

The Canada Hall in the Canadian Museum of Civilization. Depicting 1000 years of history, this was an outstanding exhibit. In addition to the familiar elements of contemporary museum design, it had just enough realism, sort of the heightened-fakery from a movie studio backlot. The open-ended design enabled an immersive stroll through recreations of the past. As you wandered you could go in and poke around stores, schools, mills, airports, and so on. It was almost like being in a holodeck, strolling through time (and from east to west).

Building environmental control module or splash screen for circa 1974 television news magazine?

Recreation of the Robert Frank image used on the Rolling Stones’ Exile on Main Street cover.

Stickers stuck to poles outside of the National Gallery. I took the picture without knowing what the heck I was looking at, though. It was only when I got my ticket for the National Gallery and was asked to put the same sticker on my clothing did I realize how those poles ended up like that.

That about covers it.

Here is the last post with highlights of my trip to Australia (see parts 1, 2 and 3, previously). Meanwhile, all my pictures are making their way to Flickr.

At the tram stop for the Children’s Hospital, the platform is filled with various cute “Dr. Jake” signs. I especially like transforming the transformer box into a height chart. Someone is thinking broadly about reframing the hospital from scary into welcoming and using the first point of entry – the tram stop – as the place to begin doing that.

Patience is a virtue. A post-design attempt to mollify confused users.

I could not figure out what the heck “Teady” referred to on this first truck. It wouldn’t be my first time encountering a word I didn’t understand (see “showbag” in an earlier post, say). But no, the adjacent truck says “Steady.” I went back and checked – did the initial S wear off? No, it was never there to begin with. The design – sans S – fits perfectly in the designated area on the first truck.

Bold language boldly presented.

Once again, weird people sculptures.

Free WiFi at the train station. Nice visual reference to many memes (e.g., Star Wars kid, the dancing baby, keyboard cat) as well as Facebook and Twitter.

ACDC lane. Yes, the official City of Melbourne lane commemorating ACDC.

Reminded me of a similar sign in San Francisco.

The menu at this restaurant offered beer in a pot or a beaker. I was further charmed by the use of familiar words with shifted meanings, as part of my foreigner’s journey. But no, the beaker is exactly what you’d think: a beaker.

More 8-bit-videogame streetart, although Mario instead of Space Invaders; stickers instead of tiles.

Many many flavors of hot chocolate at Chokolait. I must go back!

Indoor nighttime climbing wall.

I got back last week from two weeks in Australia, traveling around as well as speaking at UX Australia and Service Design Melbourne. Here is the second of four posts with some of the highlights. Part 1 is here.

All my pictures are making their way to Flickr, as well.

Nice combination of cultural iconography.

I liked seeing the range of medical services laid out like this. Not very confidence inspiring, however.

Manners poster.

Asian preferred.

A really awful brand name and sign. What does crocodile have to do with thai? And what is added by making him or it senior? It’s a puzzle.

Father’s Day is September 2.

Milk for the workplace. At last.

360˚ Self-Portrait at the Museum of Contemporary Art. Although there are no visual cues, the artist is in some contraption that moves her around, through a complete circle. As she moves, gravity deforms her face slightly while she essentially remains expressionless. The net effect is compelling and disorienting.

The 18th Biennale of Sydney was chock full of astonishing spectacle. Really wonderful. And I only heard about it because I checked in (via Foursquare) at the museum and a kindly person on Twitter suggested I head over to the island and see it.

A crushed car was on display and yet needed to be marked before and after with warning signs so that traffic going by would not be alarmed. Are the signs part of the work or something that is imposed on the artist in order to allow their work to proceed? And what does it mean to have a sign announcing “End Artwork” anyway? Is that an observation or an imperative?

Anthropologist Francoise Brun-Cottan, an ethnographer on the WorkPlace Project team, recounts a story about embedding deeply in an active work environment in comparatively primitive time, both in terms of the recording equipment and the field’s sophistication in describing to participants what was happening and why.

It was the winter of 1989. Members of Lucy Suchman’s group at PARC embarked on a multi-year joint Steelcase/Xerox project to look at ground control operations of two airlines at San Jose airport. Airport management and each of the airlines’ managements were “on board.” The project would study ground control operations of each airline in the existing airport facilities and then follow them over to newly built facilities in a new structure.

We were going to look at the interplay of paper (manuals, computer printouts), voice (over the air, walkie-talkies, radio, and telephone), chalkboard and whiteboards, and direct visual observation (versus camera/video feeds) of planes pulling into and out of the gates and being cleaned, fueled, and having baggage loaded and unloaded.

The plan was to use basic ethnographic methods and techniques: interviewing, shadowing (inside and on the tarmac), audio and video taping, still photography, and transcribing recordings to…we weren’t quite sure exactly what we’d come up with. But we wanted to find out what could be extracted using these methods in such a tech-heavy environment. We were betting it would be informative, insightful, and valuable to everyone concerned. We hoped.

In 1989, technology had come a long way from the cameras and recorders that had to be moved about on dollies and trailers but, as portable as they were, the wonderful Panasonic video cameras were heavy, and so were the VHS tapes and batteries. The little Sony audio tape recorders (about a the thickness of a stack of 10 iPhones) were great for interviews but micing a control room took hours. Synching the mikes with the video was a cabalistic art colorfully augmented by below-the-breath expletives.

The members of the airline ground crews had basically been informed by management that they were going to cooperate. In reality that mean they were going to “tolerate”: non-lingo literate researchers climbing over their equipment whenever they weren’t actually underfoot, being recorded for hours at a time (rather gleefully when crises were underway) and being followed around and asked question after question (further confirming the depth of the researchers’ lack of common sense and basic knowledge) whenever there was a moment of down time.

We did try and tell crew what we thought we were doing. But saying that we wanted to understand their work practices, how they would change in new environments using new technologies and how they made sense of their work and communications was not exactly revelatory. As a field, we’re much better at doing that now, partly because we’ve also got lots more examples we can show to prospective participants about results of our work.

Viewing the videos made clear what some of the crew members stated directly, which is that they were deeply suspicious of what we would report about their actions to management. Anytime they deviated from protocol, or made mistakes, or seemed to be resting rather doing some piece of work might be opportunities for management reprisals of one sort or another. Some people were openly, if politely, hostile. No one welcomed us. It was tough on participants and researchers alike. They always cooperated at some level; there was no point in antagonizing us (though some were pretty gruff).

It is known that after a time people seem to forget that cameras are rolling, even if those cameras are right in front of them and so are the researchers. Viewing the weeks of tapes gave us plenty of opportunities to see them cast black glances our way, or whisper something together and laugh at us. In their banter we also learned about their home lives and romances and trials and aspirations. Sometimes that could let us congratulate them, sometimes our knowing was resented.

I don’t remember exactly when it became clear to our team – maybe somewhere toward the end of our fieldwork – that the final report would be a 2-hour video tape, which came to be known as The Workplace Project. We said our good-byes and appreciations on the last days of fieldwork. “Thanks,” they said and (figuratively) “good riddance.”

We made sure that copies of the tapes were made available to the crews at both airlines. We learned that the report was being used by the crews to convey to mid –management complexities about the work that the crew members had not been able to convey. They repeatedly mentioned the benefit of demonstrating how manuals misrepresented events and complicated the work rather than facilitating it. Displaying the degree of expertise needed to differentiate personal task-relevant details from a sonic soup of incoming streams of information highlighted their previously unacknowledged proficiencies.

Sometimes you just have to stick with it, whatever “it” will turn out to be. And then, sometimes, you get to get thanked for revealing aspects of the work that the workers can’t make visible. A few years ago I was told that Steelcase still showed the video to visitors to their user centered division. So, that’s not too bad!

Here’s another proof point for the power of video in user research. Check out this very simple observational video.

If you didn’t watch it, it shows person after person stumbling on poorly designed stairs.

I don’t know about you but I felt increasingly emotional the more I watched this. A bubbling outrage and a sense that something so obviously needs to be done about this. Of course, this is a simple problem, which makes the failure to act even more aggravating.

The goal of user research isn’t always to uncover people’s fail states with the team’s existing products, but when it is, tools like video are impactful on rational and emotional levels.

Update: according to this Tweet, the stairway is now closed.

More observations from last week’s trip to Lisbon. See part 1 here

Street art.

Body-enhancing undergarments.

Clooney.

Eat box? Yum!

Y’arr! Pirate Bar! That’s some great neon. Perfect place for Drink Like A Pirate Day.

Scented dolls? They look pretty intense. Perhaps they inch forward menacingly as you pass by their window.

The design museum was redoing its facade with sticky notes. We watched their progress over several days.

This is the take-a-number device for a retail queue. Far more advanced than the familiar North American paper ticket dispenser. And also unrecognizable if you don’t read Portuguese and don’t know to look for this.

Detail of a building exterior. Tiled buildings are ubiquitous, with many different beautiful tile designs of various vintages.

Last week I had the pleasure of visiting Lisbon and presenting at UX Lx.

I gave an updated version of “Well, We’ve Done All This Research, Now What?” where we did a brief observation of the area around the venue and then developed concepts that spoke to the needs we uncovered. Among the concepts the teams played with was a giant robotic sheep that would provide shade.

The slide deck:

Per Axbom took a series sketchnotes during the session and kindly posted all of them here.

I gave a short presentation on the final day of the conference, exploring the power of user research not only to uncover data that drives product development but to change the way an organization thinks about it’s customers and itself.

The slide deck:

Sketchnotes from LiveSketching.com, Per Axbom, and Francis Rowland. Click on any of them to see the larger original.

(Side note: amusing to see the consistent use of the presenter caricature. The organizers of the conference may have contributed to this; in each attendee packet was a poster showing a funny if awkward scene with cartoon representations of all the different speakers, as well as a set of cards for one of the speakers. Attendees were supposed to trade cards until they got a complete set.)

We’ve had a lot of good posts – and comments – as of late about extroversion, introversion, talking to strangers, comfort zones, and so on. This brought to mind a story from a visit to New York a while back. In Let’s Embrace Open-Mindedness I tell two stories from my personal life (e.g., not when conducting research) where I explored the edges of my own comfort zone in just slightly unfamiliar circumstances, one situation where I saw the opportunity and couldn’t make the leap, another where I saw the opportunity and convinced myself to take that leap.

Followers of this blog will know I love taking pictures of curious and interesting things that I see everywhere, but it’s much harder – and not always appropriate – to take pictures of the curious and interesting people that I see everywhere. Indeed, in true Heisenberg fashion, you can’t always get the picture you’d want if you have to interact.

Anyway, visiting New York and walking through Times Square, I came upon people promoting Carrie Fisher’s “Wishful Drinking.” They were dressed as parodies of the book cover, with Fisher as Leia. At two separate points, I asked these hawkers if I could take their pictures.

Well, sure. It’s New York. It’s Times Square, thick with tourists, and these people are calling attention to themselves for promotion. All those cues shift the norm and make it reasonable/comfortable/appropriate/possible to do something that we don’t normally do: asking “Hey, can I take your picture?” That’s probably why I have so many photos taken with Shrek, Mr. Peanut, an Animaniac, the Monster.com monster – there’s something delightful and ironic about this staged naturalism, as if yes, I am hanging out here with my arm casually thrown around a 6-foot be-monocled legume. The opportunity to ask for a picture is so built-in to our scripts that it seems a crime to not get the picture!

Also see: The bear that saluted me

I visited the great city of Portland, Oregon over Thanksgiving week, and noticed some of the ways its denizens use surfaces to communicate and express. Like Steve did earlier in his recent post, Out and About: Steve in Boston, given our recent interactions article about noticing and documenting street art, Kilroy Was Here, I too wanted to share some snaps!

As elsewhere, the backs of city signage serve as canvas for quick-stick expression. The tiki-figure here is one I commonly see in the Mission neighborhood of San Francisco, where I live, which surprised me and gave me a little charge, a feeling of connection to home.

A City of Portland sanctioned sticker, which includes a number to call to report damage to this sign, sits alongside its renegade brethren.

Great juxtaposition of two messages about the dangers of inhaling alongside a DANGER sticker.

I appreciated the friendly, bubbly, colorful style against the rainy, grey backdrop of Portland. Contributors to the collective urban collage here seem respectful of each others boundaries – not much overlapping of images.

And, finally, bunnies!