Posts tagged “feedback”

How not to solicit customer feedback

comments

Seen at a busy lunchtime eatery in San Mateo. The comment card box is placed right above the full garbage bin. No comment cards or pencils on hand, so perhaps this is just a vestige that they haven’t bothered to remove. But abandonware always makes you look like you just don’t care and in this case the visual association between “we value your opinion” and “food waste” is not appealing. Greatly appreciated? I doubt it.

Adventures in Consumption

Here’s a bunch of examples of surprise, delight, dismay and beyond from my recent interactions in the consumo-sphere.


From the travel section at The Container Store. Lots of fun little bottles for packing your unguents and potions for travel. Nalgene bottles are guaranteed not to leak, even in the unpressurized airplane cargo hold. Given that the most you can carry onto a plane under TSA regulations is 3 oz., that seems like a likely size. Nalgene doesn’t make that size, despite sufficient demand that The Container Store has printed up a special sign to try and deflect the inevitable inquiries. What are those conversations like between the Nalgene sales rep and the buyer from The Container Store?




Paying online for the San Francisco Chronicle. In addition to the cost of the paper, I can also add a tip for the carrier, or donate some money to NIE. Not a misspelled Monty Python reference, it’s Newspapers in Education (I Googled). You’d imagine they’d get more uptake if they told us what it was they are asking for money for.




From the San Francisco Museum of Modern Art. “No peeking” (and “come back”) is so much nicer than “keep out.” And so knowing; of course when you see an installation-in-progress you are curious! The SFMOMA acknowledges that curiosity and harnesses the energy behind it to encourage you, rather than discourage you.



The menu at Oyaji in San Francisco. We see the risk of software that uses default form entries when you end up with Spider Roll that consists of “Give a brief description of the dish.”



At Crate and Barrel, shoppers can send a text to the manager to give feedback about their shopping experience. I hadn’t heard of this service (from recent Google acquisition TalkBin) before.



A travel poster advertising Alaska. And bears. Funny, friendly bears. Who, if you read the news, keep eating people.




A poster from a local cafe advertising Elizabeth’s range of services. I’m not sure I’ve ever seen a more tangible demonstration of the importance of specializing in your positioning. While I’m sure Elizabeth is wonderful and if I got to know her I’d trust with everything including yard maintenance and meal preparation, but to a new customer, someone who is qualified to look after precious offspring isn’t therefore qualified to look after precious animals (and in fact my be less qualified…do you want your toddler in a house full of someone else’s dogs?). Pick what you are good at and sell the one thing. If you need to diversity, create a range of separate messages.



Rooms at the Edgewater Hotel in Seattle have lovely specialized bottles of hair care products that reflect their brand and overall attitude. Unlike most hotels with their tiny (3 oz.) sample bottles, these are big, easy-to-handle bottles like you might have at home. A sign warns you that it’ll cost you $25 to take them home, so you know it’s good stuff. Mind you, on the housekeeping cart are these ketchup-and-mustard-evoking-bottles with stick-on labels that are used to refill those lovely bottles. Delightfulness denied.



Pike’s Place Market in Seattle. Past the faulty grammar (How the elephant got in my pajamas, I’ll never know!) the motivation for this extreme warning is clear enough.



The ice cream menu at Cold Stone Creamery. Random, unfunny, unintegrated product name puns. One evokes James Bond, but why? None of the others do. Other names are silly but decidedly not clever. My favorite is Cookie Minster, made with mint, so you’d think it’d be Cookie Mintster but no. Not that.

ChittahChattah Quickies

  • [from steve_portigal] End Of An Era: Sony Stops Manufacturing Cassette Walkmans [Crunchgear] – [I share the author's surprise that this product was still being manufactured! The CD Walkman – its successor – has long been quaintly outdated, so cassettes? Perhaps there was a retro market, or perhaps other countries discarded formats differently than we have here] Sony announced it will stop manufacturing and selling these devices in Japan – after 30 years. Sony says the final lot was shipped to retailers in April this year, and once the last units are sold, there will be no cassette Walkmans from big S anymore. The first Walkman was produced in 1979. The TPS-L2, the world’s first portable (mass-produced) stereo, went on sale in Japan on July 1 that year and was later exported to the US, Europe and other places. Sony says that they managed to sell over 400 million Walkmans worldwide until March 2010, and exactly 200,020,000 of those were cassette-based models.
  • [from steve_portigal] PlumWillow Is Making the Customer Part of Its Culture [NYTimes.com] – [Employment criteria: do you represent our target customer? Hiring for insight as an internship strategy] They’re part of a team of 15- and 16-year-old interns who are being tapped for their own special brand of expertise and insight: a bird’s-eye view into the life and mind of high school teenagers, exactly the audience that PlumWillow is seeking. “They definitely aren’t shy about telling us what they like and don’t like,” says Lindsay Anvik, director of marketing at PlumWillow, who helps oversee the internship program at its offices in Manhattan. The interns are also emblematic of how Web-based businesses are doing more than merely shaping their products and services around customer preferences. The companies are corralling those customers in the workplace and making them part of the design and marketing process, according to Susan Etlinger, a consultant at the Altimeter Group, which researches Web technologies and advises companies on how to use them.

ChittahChattah Quickies

  • [from steve_portigal] DarkPatterns.org – [This site seems aimed at designers but could also be the seed of a User Literacy effort to raise awareness among consumers] This pattern library is dedicated to Dark Patterns: user interfaces that have been designed to trick users into doing things they wouldn’t otherwise have done. Normally when you think of “bad design”, you think of laziness or mistakes. These are known as design anti-patterns. Dark Patterns are different – they are not mistakes, they are carefully crafted with a solid understanding of human psychology, and they do not have the user’s interests in mind. The purpose of this site is to catalogue various common types of Dark Pattern, and to name and shame organizations that use them. [via @kottke]
  • [from julienorvaisas] How to shrink a city [The Boston Globe] – [The shrinking economy has forced a new way of looking at strategic planning and innovation in the housing and urban planning sector.] “It’s so contrary to what most planners do, it’s contrary to what we spend our time teaching students, [which is] all about how do you manage growth and accommodate growth,” says Joseph Schilling, who teaches urban affairs and planning at Virginia Tech University and helped launch the National Vacant Properties Campaign. “The challenge for planning is how do you adapt existing tools and planning strategies to deal with an economy and market that is either totally dysfunctional or will have maybe slow, modest growth at best.”
  • [from julienorvaisas] Americans Demand Crispier Outside [The Onion – America’s Finest News Source] – [Alas, if only the elusive consumer would come out of hiding and just tell us what they want, nay, what they need!] Irate citizens have rallied in front of shops and drive-thru windows nationwide to outline their demands, which include extra chunks, meltier bits on top, that classic buttery flavor the whole family can enjoy, and a wider array of sizes, shapes, and colors to mix and match. Sources are also calling for cleanup to be a breeze.
  • [from julienorvaisas] What If Google and Bing Waged a Search War and Nobody Noticed? [Advertising Age – DigitalNext] – [Full of quippy critiques of the nutty design evolution of search, reviews, online advertising from a "real person's" perspective, this slightly ranty column by Kevin Ryan is really a lament to how beholden so many of our experiences are to today's digital monoliths.] Instant search is another one of those solutions created by engineers completely out of touch with humans. Like instant coffee, it sounds like a good idea until you have to consume it. My guess is boredom and fatigue from all that free food and the happiest work environment on the planet has finally taken its toll. In other words, idle hands solve problems that don't exist.

ChittahChattah Quickies

  • [from julienorvaisas] New Artisanal Pencil-Sharpening Project [Details Magazine] – [It looks like the artisanal food and craft movement may be fading in cultural relevance if it's subject to this level of brutal skewering.] "What better to complement your collection of limited-edition notebooks, small-batch liquors, and locally sourced honey than a pencil sharpened by a true artisan? David Rees, author of the comic book series Get Your War On and My New Fighting Technique is Unstoppable, discovered his passion for sharpening pencils while working for the U.S. Census Bureau. Now he's parlaying his old-school skills into a mail-order artisanal pencil-sharpening business."
  • [from steve_portigal] An App for ‘Despicable Me,’ to Use at the Theater [NYTimes.com] – [Is there a difference between multimedia enhancement and advertising-supported distraction?] Best Buy Movie Mode is being released in connection with “Despicable Me,” an animated 3-D movie in which an aspiring supervillain named Gru inherits three little girls. The marquee feature of the app is called the Minionator, which translates the gibberish of Gru’s little yellow henchmen called Minions. In theaters, the Minionator will work only during the closing credits, but on Blu-ray disc throughout the movie. “It is disturbing to have people doing things that take people out of the movie,” said Patrick Corcoran, director of media and research for the National Association of Theater Owners. Many theaters warn patrons to turn off their phones. Movie Mode tries to appease those who dislike distractions. The app automatically turns off a phone’s ringer and dims the screen to discourage texting. It does not disable the phone. It will still vibrate.
  • [from steve_portigal] Black Taxis offer tours of Belfast [SF Chronicle] – The Black Taxis of Belfast grew out the height of the Troubles. City buses were subject to bomb and sniper attacks as they passed through the strife-torn neighborhoods. Safe passage had to be arranged via taxi, and the taxi drivers could only operate within, never across, each neighborhood's boundaries, The ads for Black Taxi tours promise a neutral historical narrative. That's a tall order, as many drivers have a genuine history on one side of the conflict or the other. Some lost family members. Everyone lost friends. Still, the mere fact that the murals are now a tourist attraction, rather than a touchstone for violence, may signify that peace has actually arrived in Belfast. "We debated whether to encourage this trend or to downplay it," said Bernard McMullan, a representative of Tourism Ireland, of the popularity of the Black Taxi tours. "But in the end, we decided that it was an important part of our history. There's no point in denying it. Besides, it's interesting."
  • [from steve_portigal] Nissan adds noises to Leaf electric vehicle as safety precaution [WaPo] – [The design challenge of creating new, yet familiar feedback cues] After exploring 100 sounds that ranged from chimes to motorlike to futuristic, the company settled on a soft whine that fluctuates in intensity with the car's speed. When backing up, the car makes a clanging sound. Nissan says it worked with advocates for the blind, a Hollywood sound-design company and acoustic psychologists in creating its system of audible alerts. Nissan's sound system is the first created by a major manufacturer. The company says it is controlled by a computer and synthesizer in the dash panel. The sounds are delivered through a speaker in the engine compartment. A switch inside the vehicle can turn off the sounds temporarily, but the system automatically resets to "on" at the next ignition cycle. At speeds greater than 20 mph, any car, electric or not, makes significant noise because of the tires slapping on the pavement, engineers say. The noises for the Nissan operate only at the lower speeds.

ChittahChattah Quickies

  • Gerry Gaffney interviews David Hill of ThinkPad [UXPod] – We try to get as much user feedback as possible, and we have many different ways we can gather that. For instance we have advisory councils, we have analysts and things of that nature comment on future plans. We meet with large customers and disclose to them future plans and get their feedback on our products. I use my blog, very effectively I believe, by occasionally posing questions, doing polls or even much more detailed web surveys. So really there's not one answer to how you get feedback from customers. But my belief is that there's no such thing as too much information. I love to meet with a customer, or observe people at an airport or in an airplane or in a classroom. You'd just be amazed what you can learn from field research or in a conversation with a guy next to you on an airplane. It's just remarkable. Sources really are so varied that it's difficult to put your fingers on [and say] this is the process we used to gather feedback.

Effective Concept Testing: Getting the Answers You Want to Hear!

We were intrigued to see that Netflix is soliciting customer feedback about a new product concept. It’s great to see them incorporating users into the development process, but we figure if they are going to be asking these sorts of questions, they might want to take the next logical step. Check out our re-enactment:

ChittahChattah Quickies

  • Social science meets computer science at Yahoo [SF Chronicle] – Yahoo Labs has bolstered its ranks of social scientists, adding highly credentialed cognitive psychologists, economists and ethnographers from top universities around the world. At approximately 25 people, it's still the smallest group within the research division, but one of the fastest growing.

    The recruitment effort reflects a growing realization at Yahoo that computer science alone can't answer all the questions of the modern Web business. As the novelty of the Internet gives way, Yahoo and other 21st century media businesses are discovering they must understand what motivates humans to click and stick on certain features, ads and applications – and dismiss others out of hand.

    Yahoo Labs is taking a scientific approach to these questions, leveraging its massive window onto user behavior to set up a series of controlled experiments (identifying information is always masked) and employing classic ethnography techniques like participant observation and interviews.

  • Domino’s "The Pizza Turnaround" [YouTube] – Domino's Pizza uses customer research to spawn product redevelopment, and then uses that process to promote their improved product. Note the negative quotes posted on the walls of their office.

Son of Survey Madness

We’ve posted any number of survey design critiques over the years, and here’s the latest, a close read of a question and the cues associated with different responses.

In response to the prompt How closely do you agree or disagree with this statement: “We saw business strengthening in the Spring, but it seems to be stagnant or falling off again. We thought we had seen the bottom, but now we are not sure.” we’re asked to move a slider between Agree Completely and Disagree Completely.
smiley
frowny

As we move the slider, the expression on the little green character changes, supposedly to provide an additional cue to ensure that our response is accurate.

But when we agree (a positive emotion), the guy is frowning. Because we are agreeing with a negative in which case we making a negative observation? So we feel negative? But the green dude isn’t mapping our feeling about the situation, he’s mapped to our response – our degree of agreement. We can feel positive about agreeing, even if the thing we agreeing about is negative (haven’t you ever exclaimed enthusiastically at someone that expresses a similar frustration to you? That’s being positive about a negative). The mapping here is wrong.

It’s further complicated by the indirectness of the prompt – that situation you are agreeing or disagreeing with – describing a situation going from positive to uncertain. How much do you agree or disagree with: something was positive but now it’s negative? In fact, besides being indirect and somewhat abstract, it’s also a compound question. You might agree that things were positive, or you might now. You might agree that things have gone downhill, or you might not. The question is asking you to agree ONLY to the cause where i) things were positive and ii) things have gone downhill. If you don’t agree with both of those, then what do you do? And since you can indicate the strength of agreement/disagreement, how will people interpret the question? I would suggest not very reliably!

Ironically, this is a survey aimed at providers of market research services, who should absolutely know better.

ChittahChattah Quickies

  • The Nike Experiment: How the Shoe Giant Unleashed the Power of Personal Metrics – Using a flood of new tools and technologies, each of us now has the ability to collect granular information about our lives—what we eat, how much we sleep, when our mood changes.
    Not only can we collect that data, we can analyze it, looking for patterns, information that might help us change both the quality and the length of our lives. We can live longer and better by applying, on a personal scale, the same quantitative mindset that powers Google and medical research. Call it Living by Numbers—the ability to gather and analyze data about yourself, setting up a feedback loop that we can use to upgrade our lives, from better health to better habits to better performance.
    Nike has discovered that there's a magic number for a Nike+ user: 5. If someone uploads only a couple of runs to the site, they might just be trying it out. But once they hit 5 runs, they're massively more likely to keep running and uploading data. At 5 runs, they've gotten hooked on what their data tells them.
  • To Sleep, Perchance to Analyze Data: David Pogue on the Zeo sleep monitoring system – Just watching the Zeo track your sleep cycles doesn’t do anything to help you sleep better. Plotting your statistics on the Web doesn’t help, either.

    But the funny thing is, you do wind up getting better sleep — because of what I call the Personal Trainer Phenomenon. People who hire a personal trainer at the gym wind up attending more workouts than people who are just members. Why? Because after spending that much money and effort, you take the whole thing much more seriously.

    In the same way, the Zeo winds up focusing you so much on sleep that you wind up making some of the lifestyle changes that you could have made on your own, but didn’t. (“Otherwise,” a little voice in your head keeps arguing, “you’ve thrown away $400.”)

    That’s the punch line: that in the end, the Zeo does make you a better sleeper. Not through sleep science — but through psychology.

  • Baechtold's Best photo series – While they are framed as travel guides, they are really more visual anthropology. A range of topics and places captured and presented in a compelling and simple fashion, illustrating similarities and differences between people, artifacts, and the like.
  • It's girls-only at Fresno State engineering camp – This is the first year for the girls-only engineering camp. Its goal is to increase the number of female engineering majors at Fresno State, which lags behind the national average in graduating female engineers. Nationwide, about 20% of engineering graduates are women. 20 years ago the national average was 25%. At Fresno State, only 13% of engineering graduates are women.

    Jenkins said he hopes the camp will convince girls "who might not have thought about it" that engineering is fun, and entice them to major in engineering.
    (via @KathySierra)

  • Selling Tampax With Male Menstruation – This campaign, by Tampax, is in the form of a story featuring blog entries and short videos. The story is about a 16-year-old boy named Zack who suddenly wakes up with “girl parts.” He goes on to narrate what it’s like including, of course, his experience of menstruation and what a big help Tampax tampons were.

Get our latest article, Ships in the Night (Part II): Research Without Design?

prototype
My latest interactions column, Ships in the Night (Part II): Research Without Design? has just been published.

Our client had the right idea-get feedback on something unfinished in order to improve the finished product. Unfortunately, aspects of the object were so unfinished that people were unable to make the leap from the prototype (excuse me, appearance model) to the real thing, and the outcomes shifted away from usability and aesthetics toward high-level concept validation. Given that, there’s always the opportunity to create something specifically to provoke people around the deeper issues we want to explore.

Get a PDF of the article here. To receive a copy of the article, send an email to steve AT portigal DOT com and (if you haven’t given us this info before) tell us your name, organization, and title. We’ll send you a PDF.

Be sure to read Ships in the Night (Part I): Design Without Research? as well.

Related: Steve Portigal speaks at User Research Friday – Design and Research, Ships in the Night?

Other articles

ChittahChattah Quickies

  • Listening to customer feedback? Twenty-Five Years of Post-it Notes (Thx, @susandra) – In '77, 3M decided to test-market. It failed to ignite interest. “When we did the follow-up research, there just weren’t a lot of people saying this was a product they wanted.”
    "We knew the test markets failed, but we just kept saying, ‘Maybe it was us. Maybe we did something wrong. Because it couldn’t be the product—the product was great.”
    To see for themselves how people responded to Post-it Notes, 2 execs cold-called offices, giving away samples and showing people how to use 'em. The responses were more enthusiastic. “Those things really were like cocaine. You got them into somebody’s hands, and they couldn’t help but play around with them.”
    1 more test was in order. They got newspapers to run stories about it. They festooned stationery stores with banner displays and point-of-purchase materials. 1000s of samples were sent to office managers, purchasing agents, lawyers, etc. People demonstrated it to potential customers. It was a huge success, and 3M decided to launch Post-Its.
  • Listening to customer feedback? Peter Arnell Explains Failed Tropicana Package Design – Big outcry over the Tropicana packaging design (which this suggests was NOT tested but that's hard to believe) led to a return to the previous packaging.
  • Listening to customer feedback? Malcolm Gladwell on the Aeron chair – The Aeron chair was originally despised and deemed ugly. It didn’t catch on for 2 years, and then it quickly became the most popular chair. Everyone came to love it. Gladwell concludes that people find responses about some topics extremely difficult to articulate. While they may think they dislike something (like the Aeron chair), in their hearts they may actually like it. There is a disconnect that causes people to express dislike in their heads while they actually like it in their hearts (and vice versa).
  • Listening to customer feedback? Hate Facebook's new look? You'll like it soon enough. – Slate advances the point that people react to change negatively but eventually get used to the change and make it work.
  • Listening to customer feedback? Problems With NBC’s ‘Parks & Recreation’ – When do you listen to negative feedback and when do you follow your vision? I think there's an important middle-ground that is often ignored: understanding what lies beneath that feedback and choosing carefully if and how to respond to it, or how to create supporting activities that help get over the barriers that the rejection points to

ChittahChattah Quickies

  • Report: Real-world police forensics don't resemble 'CSI' – Even before the popularity of shows like CSI, there was presumably a cultural belief in the "science" behind these techniques. But the report finds that:
    – Fingerprint science "does not guarantee that two analysts following it will obtain the same results."
    – Shoeprint and tire-print matching methods lack statistical backing, making it "impossible to assess."
    – Hair analyses show "no scientific support for the use of hair comparisons for individualization in the absence of (DNA)."
    – Bullet match reviews show "scientific knowledge base for tool mark and firearms analysis is fairly limited."
    – Bite-mark matches display "no scientific studies to support (their) assessment, and no large population studies have been conducted."
  • NJOY electronic cigarette – Looks like a real cigarette, complete with glowing tip on inhale, and exhaled vapor that resembles smoke. Gives an inhaled nicotine experience, while messaging to the rest of the world that you are really smoking a real lit cigarette. Paging Erving Goffman?

    Someone was using one a party last week; someone else got out their simulated Zippo lighter (an iPhone app) and lit it for them.

Supermarket tales

I’ve been doing fieldwork for the past couple of weeks, which often means stopping in at a variety of grocery stores for quick bites to eat or bathroom breaks.

In making the rounds, I saw a couple of things I thought were worth sharing.

are-you-in-uniform3
Andronico’s, Berkeley

I thought this was an interesting way to extend the function of the mirror, and a good reminder of how much more you gain from feedback when it’s deployed at just the right time and place in a process.





cheeses1
Whole Foods, San Francisco

This was without question the most fragrant cheese counter I’ve ever encountered. I was standing with my back to it, looking at the fruit, and I kept thinking something was wrong somewhere. I finally turned around and understood what I’d been smelling.

Who’s thinking about the customer experience here? What would some alternatives be? Put it near the fish? Or how about near the flowers! A giant plastic dome over the whole thing? Perhaps an information station explaining why cheese can sometimes be stinky…

Communication Confusion over Confirmation Confusion

Andrew points out that Kyoto and Osaka are near each other and that was probably behind the offers from Expedia that I had complained about. Helpful info that I didn’t have, but unfortunately, it got worse.

A few days later I received email from Expedia

To: Steve Portigal
Subject: Steve, here is your itinerary confirmation for your 01/02/08 Osaka trip
[deletia that makes reference to our Osaka trip multiple times]

Even more concerned than before, I wrote them and received this message

Dear Expedia Customer,
Thank you for contacting us.

We regret that your experience with Expedia.com was not satisfying. Comments such as yours are read by numerous people within Expedia and help shape our policies and practices as we learn and grow.
If you have further questions, feel free to reply to this e-mail or contact Expedia customer services at 1-800-397-3342 and reference case ID 36793797.

In other words, I submitted a complaint and they aren’t going to act on it, unless I submit it AGAIN. Okay, I do that.

The next message is even worse.

Dear Expedia Customer,

Thank you for contacting us.

Kyoto, Kyoto-fu (Change name) Expedia.com itinerary number: 121380781812

If you have further questions, feel free to reply to this e-mail…

No actual communication. Is anyone out there? I try again.

Dear Expedia Customer,

Thank you for your immediate response.

Please accept our apologies for the misunderstanding with your hotel reservation. We regret any inconvenience this may have caused you.

Your itinerary serves as a confirmation of your purchase, and we’ve sent an updated copy of it in a separate e-mail. You can also access your itinerary online at any time. Here’s how:

Again, no one is addressing my key question: why does my Kyoto reservation keep getting referred to as my trip to Osaka? Once more into the breach…

Dear Steve,

Thank you for your reply.

Please accept our apologies in regards to the misunderstanding with your reservation. We regret any inconvenience that may have occurred and would like to assure you that every reservation is important to us.

Your problem may stem from incompatibility between your browser and our system. We have already escalated this technical issue to the appropriate department.

In the interim, your hotel reservation at the Hotel Monterey Kyoto is confirmed while the “OSAKA” tag line have caused you such inconvenience, the printed itinerary of your reservation is still binding and a confirmed reservation for a hotel in Kyoto, Japan and not in Osaka.

So somehow my browser is causing them to send me email messages about a different city? The crucial piece of info (thanks, Andrew), that these are nearby cities, never appeared, and a spurious technical issue was blamed (it’s not a browser issue; perhaps they want to blame the model of car I’m driving for the emails they are sending?) but at least a human being intervened and confirmed that what I thought I bought was indeed what I bought.

Great job, Expedia people! Ridiculously poor support to go with a rather silly system! Let’s hope we don’t have an actual problem at any point.

Series

About Steve