12. Pree Kolari of eBay
This episode features Pree Kolari, the Senior Director of Design Strategy and Research at eBay. We talk about the career arc of a researcher, having impact on the product, and breaking down organizational walls.
We can make it a little better in terms of customer experience where they buy with one less click. We can also change the dollar amount eBay makes by .1%. That’s a win and it’s very easy and it is very, very tempting for everybody on the team to go “Yes that’s great, let’s go do it!” But then you ask the why questions and go for deeper insights, not just the surface level insights. You go deeper. You find out that it’s a lot more and you will have small things that you could do on the product that will give you wins two years from now. – Pree Kolari
- Pree Kolari
- Follow Pree on Twitter
- eBay Classifieds
- eBay Motors
- eBay Fashion
- local maximum
- stack rank
- Moto X
- The Design Sprint
- Karmann Ghia
- A Pattern Language by Christopher Alexander
- The Designer is Present
- Product Management + User Experience
Steve Portigal: Thanks, Pree for being a guest on Dollars to Donuts.
Pree Kolari: Absolutely. Looking forward to talking.
Steve: Let’s do the basic intro part of the interview. Tell me something about you and your work and kind of why we’re here to talk.
Pree: Sure. I’m Pree Kolari and I lead the Design Research team at eBay. It’s actually Design Research and Strategy team that I lead at eBay. And we’re a pretty sizable team, about 20 people across the US and Europe. So we have people in San Jose, San Francisco, Portland, Germany and London. So pretty sizable team. We work across all of the products on eBay. So we work within of – we work with the product teams and work on both new products as well as existing products.
Steve: So if I’m not an industry insider and I’m listening to this and I think eBay – eBay has multiple products? Like can you explain you know eBay a little bit for those of us that aren’t in the know.
Pree: Sure, absolutely. So eBay is interesting because you have a couple of different kinds of users. One is the seller and then another one is a buyer. So both of them see different sides of eBay. And they both interact with eBay via desktop, mobile, you know multiple devices as well. So every one of those interaction spaces are product areas from our standpoint. And so that’s one. Then eBay as a company plays in many, many different spaces. One is eBay the marketplace itself. Then there is a bunch of other places which is Classified, StubHub and all of those things. Those are separate teams by itself and generally speaking we don’t necessarily work across as much as we should, but we do. So that’s – does that that give you…
Steve: So where does something like eBay Motors fit into that?
Steve: Is that like a StubHub?
Pree: It is – no, no, no eBay Motors is part of our team. Those are all things that are considered verticals. Good point. So within of – the shopping side of things – there are certain areas that have specific verticals and eBay Motors, or Fashion, you know those are the spaces that are a part of our team.
Steve: If you’re an eBay shopper do you experience those things – if you just go in and search do you know when you move between the verticals?
Pree: Ideally if we did our job well from a design perspective, as well as from a research perspective, you shouldn’t notice it. But sometimes we do, you know because it is two different teams that are delivering the same product, right, and so that’s one of the big challenges of doing good research because as a researcher you’re thinking end to end. You’re thinking the customer journey. You’re coming at it from a user standpoint and users don’t see your org structure. They see the product. The moment the users see our org structure we have a problem and that’s exactly what we shouldn’t be doing.
Steve: So it sounds like the experience today for someone, there’s lots of points where you wouldn’t see the structure and some points where you would.
Pree: Absolutely, absolutely. Lots of places where you shouldn’t see it, but some places you see it and that’s kind of why a bunch of us have jobs.
Steve: To try to…
Pree: Try to fix it. Yeah? Yeah, it is about fixing. Some of it is about fixing. Some of it is about like coming up with new things that will make the older behavior that we have started kind of become back – you know take a back seat.
Steve: It makes me think about what – you know how research fits in, which I think we could probably spend a long time talking about – I mean I think sort of the counterexample is always the – you know there’s a question that comes from a certain team and they’re really only interested in the answers that affect their decisions within that.
Pree: That’s right.
Steve: Whereas, as you say, as a researcher you’re looking kind of end to end.
Steve: So the question is you know what – you know how does the work that you do – how do you structure it to touch different teams at different points so that those decision can be made.
Pree: It’s a good question. So one of the things that our team, the whole team does is talks to our counterparts, our colleagues or people who are part of our core teams. Yeah? And the main point there is understanding the goals of what we are trying to achieve. The secondary part there is to train people not to look for local maxima. We have – we have a habit – and this is just a bad habit that’s come over time, of looking for small wins, local maximum. Instead, how can we focus on long term thinking? So there is a little bit of bringing people along that happens. And during that time the framework that we end up using is just a simple one of generative, iterative, evaluative. So here’s the different kinds of spaces that we can look at research. The first part is generative – that’s about generating ideas. That’s about diverging in from an ideation standpoint. Next one is iterative – this is where you’re continuously testing things out to learn quickly and make this product much, much better. And then there is evaluative. Now we’re pretty close to ship. How do we make sure that we ship something really amazing? And so that’s the three step kind of a thing. If you look at a waterfall process, you look at an agile process, pretty much those things fit right in. Yeah.
Steve: Can you say a little more about local maxima? That’s a great term that comes from say like some kind of number theory something.
Pree: It’s a statistics kind of a – it comes from statistics. It’s used a lot in the stock broker side of things, right, and because you’re always – there are people who are there for the short term, they just are day traders that just made money. They want to sell it off and then they want to move on to the next one. So if you look at today’s stock you probably see an up or a down and then the up could be a good up and it could be the maximum up that the stock has ever gone, or you could step back a lot and see that the trend has been going up for years – 25 years or 30 years. And then you suddenly think, okay you know what if I hold on to the stock it’s going to do very well. So now you look into a product or a research side of things. You start to see that we’ve made – we’ve taken a certain path. Let’s say with eBay we’ve taken a certain path with respect to how we make a buyer go through the categories, go through the shopping experience. And we can fix it a little bit. We can make it a little better in terms of customer experience where they feel like hey you know I got to buy with one less click. We can also change the dollar amount by which eBay makes out of this about .1%. Now that’s a win and it’s very easy and it is very, very tempting for everybody on the team, including us researchers, to go yes that’s great, let’s go do it. But then when you ask the why questions and then you ask – you go for deeper insights, not just the surface level insights – you can see patterns, but you have to see the insights. You go deeper. You start to find out that it’s a lot more and you will have small things that you could do on the product that is – that will give you wins two years from now. That’s what I mean by local maxima.
Steve: So I’m thinking about – we won’t get into any specifics, we’re talking about this very generally – I think about things like you might have other kinds of maxima like how often they’ll come back?
Steve: But then you might have sort of larger issues like the meaning. Like if you want to change this site – and I’m really just making this up – you want to change this from a transactional partner to something that becomes a support system.
Pree: That’s right.
Steve: Well now that’s about longer – that’s about shifting the overall meaning through a lot of design choices and you have to measure that very differently than did the amount in the cart go up. Is this what we’re talking about?
Pree: You’re absolutely right. It’s not just looking at a financial benefit that you can quickly calculate now. It’s about looking – it’s about taking some amount of risk which are long term financial gains because it will come back, you know, and the team will start to have an intuition of the fact that it will come back. And there is a certain – you know we’re a profit making company, right, and at the end of the day we want to make profit every day, yeah. And so it’s not about just putting – taking all of our eggs and putting it for long term wins. It’s about being very intention and saying hey you know here’s a few things that we’re going to do where it is high risk/high reward – here’s a few things we’re going to do which is absolutely beneficial and we can see the benefits right away.
Steve: You’re describing a strategy.
Steve: Or a road map.
Pree: That’s right.
Steve: Like here’s where we want to go with this.
Pree: I am. I am describing a road map and a strategy.
Steve: And you said right at the top that strategy is in the title of your group and you’re talking about research as a strategic tool…
Pree: Absolutely, right
Steve: …very much.
Pree: Right, right, right. So as in people on the team come from different kinds of backgrounds. There are people who have no research degrees, if you may. There are people with MBA degrees, right, and that’s intention. When you put these kinds of – different kinds of people together there is – there is a radical collaboration if you may. And that kind of collaboration happens because people are coming at it from different mindsets, but they respect each other’s mindsets. Yeah. And when you start seeing that happen other people also join in and I want the engineers, I want the designers, I want the product managers joining in and pitching in and making it their own idea. And that’s how you take it to ship.
Steve: So in this – this maxima example I think is a really great one. We’re sort of avoiding the pitfall of a local maxima I guess maybe is the theme here. What kinds of conversations are you having with people and how are you helping them – I’m assuming you are helping them to see that as a choice or a different path.
Pree: Yeah, good question. So you have – there is – this is where it’s a little bit of an art more than a science. And it’s a little bit of wisdom more than here’s 5 things you can do. What I look at is I think of – you know I don’t do as much – personally don’t do as much research as I used to do out in the field. I do very, very little. But what I end up doing is I end up doing research on – in the corporate side of things if you may. It’s about understanding the organizational psychology. It’s about understanding what are the needs, what are the pain points, what are the motivations of these people who are working inside of an organization. And once you understand that and you map it you can think about like why is this person thinking this way and you can still figure out that there is a motivation that works with what you’re trying to communicate and it becomes very easy then. And so I am always looking for win/win situations. It’s a negotiation at the end of the day too. So it’s a win/ win situation. Let’s figure this out and make it happen. So I don’t think I have a specific answer there, but more of a generic answer.
Steve: You make me think about my own work. I’m someone that works outside the organization. So I come in and the best engagements for me are ones where I’m partnering with somebody who has that insider knowledge and has sort of worked over time to try to find out how to be successful so that I’m not blundering it and saying I know the best way to do it. It always has to be adapted to the context. And there’s often this interesting tension – maybe tension in a good way – when you come in – and you’re newer in this role so you probably have a perspective on this. When you come in you see everything that’s wrong that needs to be changed. And then when you spend time you choose your battles. You figure out your strategy. What I see, because I’m always in the newer side of things, is you also get – I don’t want to say co-opted, it’s too strong a word – but you become part of the system. And so I have these situations where I feel a deep passion, like this is what these people need and people that are my gatekeepers are like yeah there’s maybe not an appetite for that. And it’s not to say that we then fall flat, it’s just – it’s where we put our energy. So I love what – I mean I think it’s – I love what you’re able to do. You have to be embedded. You have to build relationships. You have to kind of go over time. Without being kind of confrontational to you, do you think, have you seen it in yourself, that you’re now part of the system – you can in to sort of drive change, but then you become part of the system. What does that arc look like?
Pree: Very good point. So I’ve done like consulting work like you have for a long time. And then I moved into the corporate side of things – 2008. So been doing this for 8 years. Big change. Big difference. The advantages and good things that corporate or the consulting side comes with is they come with a fresh perspective. And you come – like when you come and work with us you come with a fresh perspective that we never would have thought of. You also come with an expertise. Yeah. So the hierarchy that has already been established within the corporation doesn’t apply to this consultant who has just come in. And so that’s an awesome thing. Yeah. However, these – the guys who come in also do not know the culture of the organization, or do not know the history of the organization. Now within the corporate structure there is researchers inside the corporate who are embedded and they work really, really hard to build the relationships to make sure that they are doing the right thing. And sometimes yes, getting embedded for too long will make them feel like hey you know I’m part of it and I cannot make any changes. But we end up, we end up really working hard not to let that happen and to make that we go back to – like within our team we have a – we build together the team with a set of principles and these are principles that everybody in the team worked together and we just kind of articulated. One biggest principle that we have is about impact and you know we write down like hey what does impact mean? The other one is about transparency. And it’s about transparency with respect to communication. Yeah. And as soon as we end up being extremely transparent we break down walls that otherwise exist within the corporation. Yeah. And it’s part of our jobs to break those walls. Yeah. And we – within my team we are not measured by how many reports you have written or whether you have finished so many projects. You are measured by the impact on the product. Yeah. So it is super important to work with the product manager. It’s super important to work with the business person. It’s super important to work with the designer to make sure that the impact is clear. And that’s when it changes our own behavior a little bit to not conform to the culture in the way you described it. Does that make sense?
Steve: It does. I feel a little skeptical and that’s not a skepticism towards you. It’s just in general sort of having observed this because in order to be successful there’s a certain amount of conforming that has to happen.
Steve: It doesn’t mean that – you know what I mean, you have to kind of conform to figure out how to choose the battles.
Steve: And there’s battles that you’re not going to choose now because you’ve found ways to be successful. And think about yourself in 2007 – I’m going to guess, you would come in and look at yourself doing this job and like shine the flashlight in some corner and say like guys you’re denying this. And now you, 2016 you is kind of saying like well no you know we have these principles and we’ve had impact and we have this track record and like we have lots and lots of success and then – you know I’m doing a little puppet show here – 2007 you is kind of saying like no, no, no. Like over here, this thing. That’s the tension that I think is…
Pree: Absolutely. So I agree with you. So if I look back in the 2008 me I would look at the 2016 me and say hey look, you’re not listening to me. You’re not listening to this part of the equation. And that’s the reason why I think – you know it’s learning, right, from a life perspective. Absolutely, so I prioritize certain things to make an impact on? I do. And everybody on my team does. And we do it because we believe in the fact that making impact is more important than making impact on all the 10 things. So let’s imagine there is a study where we saw 10 different things that are very important things to fix, okay – an evaluative study. We might choose to pick only two of the top things and some of it might be confrontational, but we might pick only two of the top things because we know that if we pick those two and go really, really hard it will help us instead of diluting our impact with going with 10 different things. Yeah.
Pree: And that’s the choices we end up making, but the choices are super important because it’s exactly based on impact. Doing the right research means talking about all the ten because those are all observed and those were patterns that were observed. At the end of the day we are not doing research, we’re doing applied research, you know, and we’re working within a product company that ships products. You cannot ship a research report. You can ship a product. So that’s what we’re going after.
Steve: Yeah, that picking the two of the ten I think is really…
Pree: It’s an art. It’s hard.
Steve: …really interesting. And it’s not like you haven’t learned those other 8 or haven’t articulated them. I don’t know do those float around kind of in the ecosystem somewhat?
Pree: They do, right. As in they might come back again and this is where – so I joined the eBay team only less than a year ago actually and our team is quite amazing. Like have been doing amazing work for quite a few years, since ’99. The research team has been there and have been doing like quite, quite a lot of good work. There were challenges in how this work was heard within the organization. There were challenges with respect to resonance on some of the outputs. And so we had to stack rank everything. So this is the engineer from me talking here. I’m all about stack ranking, drawing a cut line and saying hey, here’s the few things we’re going to go after and we’re going to go after really hard.
Steve: Is that a fancier way of prioritizing?
Steve: Was that what you’re saying or is there like a methodology?
Pree: It is prioritizing.
Steve: Okay. That’s a great phrase though.
Pree: Yeah – stack ranking?
Pree: Yeah, it is. It’s probably engineering talk, but yes. I would rather you know – because at the end of the day we are – again, concentrating on shipping, yeah. And so again I think that those are things that are coming from my background as an engineer. As a person who has like shipped multiple products. As an engineer, as a start-up person. As like – you know – coming back to if we did amazing research and there isn’t a follow through afterwards the amazing research just sat on shelf and nobody benefited from it. Because I know that in my heart, and I know that a lot of people who do this, we’re doing it not to just do research. We do it because we enjoy seeing the output of this which is the experiences that have changed. And that’s what gives us satisfaction and that’s the long term view.
Steve: Even just the idea of sort of the output of – these are significant efforts. A “research project”, whatever it looks like, it’s a significant effort, right?
Steve: And that even to get what you learned down to 10 things or 10 big ideas or 10 opportunities that takes a lot of pain. You have to leave a lot of things on the floor and synthesize and organize. And then – I’m just sort of toying with this idea you’re describing that says okay team we’ve kind of got these 10 things, let’s go at them with two. It’s very different than – and I’m sort of thinking about the consultant cycle which is, you know, it’s hey the consultants are back, here’s their thing, it’s a significant effort. It sort of ends the engagement. We have to impart as much as possible because we’re going to leave. So to get it down to 10 is really hard, but it’s like I think about in certain circumstances like what would happen if we said okay there’s two things. Yes we have a lovely Executive Summary that says like here it is on one slide, but just to really shout loud about two things.
Pree: Sure. Actually let me give you an example, okay, which is from my previous work at Motorola. So here’s the interesting thing that happened. So Google had just acquired Motorola and that’s when I joined Motorola as part of the Google, when they acquired, around that time. And one of our – one of the challenges that we got from – during that time it was like a bunch of the Google executives who joined Motorola and it was clear there was a mandate, said let’s reduce the number of phones that we’re making to very few and whatever we are making let’s make something amazing. So we did a bunch of ethnographies where we saw people taking pictures. That’s what we were looking after. We were looking at images and people taking pictures. And one thing we observed over and over again was how hard it was for people to quickly capture an image. And what all things people were doing. And this is where – you know you’ve seen this in the field. Mom holding a kid and trying to take a picture and a selfie – there’s a diaper bag in another hand. It is, it is tough, And how do you allow for that to happen? And we could have written reports and reports about this, but we went in with one phrase. We said, you know, pocket to picture in 4 seconds and one hand. That was our output from a research study that took 2 months if you may.
Steve: And you learned, just to – you learned lots of things…
Pree: We learned lots of things.
Steve: Tons of improvements that could be made…
Steve: Lots of opportunities for fixing, creating new.
Pree: Absolutely. And when we know that we want to get from a pocket to picture this much time, okay, and one handed. Now you suddenly, suddenly have, have something that the whole organization can rally around. There is a purpose. It’s a measurable goal. And what happened was with the MotoX we had a gyro – every phone has, you know has, has a sensor in it. We put in a low power sensor as well and people could just twist twice and with one hand take a picture. We even changed the UI. In any big company you could think about like how difficult it is to remove the interface. And we removed the interface to make the whole screen a tough target and you could take a picture. And suddenly the user experience completely changed. And so could we have gone with like hey the pictures that were taken in a dark bar were too dark? Sure. We had a whole bunch of things, right, but we didn’t. We made a choice and we made multiple choices afterwards and so the quality of the picture came up after and we worked very hard to change the quality of the picture and we have specific things about how you can change tones based on the picture, like a skin tone vs. food vs. the sky and make the perception of that picture to be higher quality. And so what we’re talking about is very data driven design. And so when you have data and you’re very specific about how we can use the data to change the design it always works.
Steve: And I think it’s interesting that you research output, if you will, was sort of a vision statement…
Pree: That’s right.
Steve: …you know for a design brief. And that’s different than personas or journey maps. It’s not about the format of the output. It’s not even “Here’s what we learned about people.” You’re kind taking it all the way to say that we did this research and it tells us that this is what the product needs to deliver.
Pree: Yeah. It’s an objective. It’s, yeah it’s an objective.
Steve: It seems like there’s something interesting about how teams are structured, or how these programs kind of run, that lets a research project output a product decision. As opposed to researchers learned this, they talked to designers, they ran a session, the designers decided – you’re sort of putting it all under the same umbrella. We did this activity, the thing that we learned is the product should do “x”.
Steve: So how do you structure – how do you bring the right players together and sort of have the whole endeavor such that that can be the output?
Pree: Everybody who’s on the team is assigned to a product area. So they have a core team that they work with and it works very well in an agile kind of an environment in eBay because otherwise you’re a service organization within – of a big company. We’re not a service organization within a company. We are a product organization. So what that means is let’s imagine a person on the shopping experience. They’re part of the shopping experience team and are embedded within the team and they work with the team everyday on daily stand-ups. Yeah. And they figure out, they have – they maintain a roadmap of what research has to be done. They bring up things within the team and talk about what are hypothesis, what are myths, what are things that we know, what are data, what are insights that we know, that we really need to go after. And so all of those things are super important coming from the researcher’s standpoint.
Then the next step is about participation within the research. So I’m a firm believer that research isn’t done only by researchers. Everybody is a researcher. It’s like everybody is an artist. Like you know we all have this innate ability to observe and make conclusions. That’s part of what we do. Some people just have been trained in this and are probably much better at the structure side of it. Now you have –when you have researchers who are catalysts within the core group, to understand more about the user so that we can make products that are absolutely relevant to the need of the user. The need may be even unknown. You know it might be a blind spot. They might not even know that they need it. Fine. But building that sensitivity for the whole team is part of the researcher’s job and then it is also part of the researcher’s job to collaborate, to get to an output. When I gave you example of phrase that came in the end it is not the researcher who comes up with that kind of a thing. It happens because the designer, the engineer, the PM is all part of that discussion. They own it. They co-own it. And that’s what – that’s when you get successful products.
Steve: That’s a good explanation of sort of how you set up those conditions to make those kinds of big impact. I mean it’s back to your point of impact earlier on. If you can come up with that phrase as your output you’re setting yourself up for impact in a really…
Steve: …massive way.
Pree: And that goes a little bit – you know with research we end up – we end up thinking a lot about outputs from only one research report or one research study. That’s I think history. I really believe that it’s about building criteria, putting together principals that kind of collect all the knowledge that we know from multiple studies. And that’s part of the job of an in-house researcher, right, which is different from somebody who is coming at it from a consulting standpoint. And what that does is it helps make good decisions throughout the product development process. We’re making – you know I really believe that the intuitive decision making is bullshit. It’s either data driven decision making or principle based decision making. And so we have to give the ability for people to make the right decisions and that’s – you know it’s getting the data and putting together principles that help us make the decisions. Does that answer your question?
Steve: Yeah. So to try to tie together a couple of things that you’ve mentioned. This idea of everyone is a researcher and then different decision making styles, I’m curious about, you know there’s people that are researchers for a reason. They have kind of an approach. Can you compare and contrast maybe the mindset of someone that has researcher as their title versus. these other people on the team that they’re working with?
Pree: Sure. So I think one of the big things as a researcher, you know we are curious. And the curiosity is what makes a good researcher. The hunger – it’s a sponge. It’s like wanting to learn. And we’re paid to be unbiased. Versus let’s talk about a product manager or let’s talk about a designer. They’re paid to be opinionated. And having opinions and having ideas of where they want to go is what they’re paid for. And now there is a tension. And both people are coming at it from different angles. Both want to understand what the users want, yeah, because we want to finally make a product for our users, but are coming at it from two different angles. And so that tension is what I think is the root of the issue and then like figuring out how to manage that tension – it’s never going to go away. It’s managing it and directing it in the right direction.
Steve: And I think that tension – as you described sort of how that team, your team, came up with – now I can’t get it right – what was the – from pocket to photo in…
Pree: It doesn’t matter.
Steve: …four seconds. Whatever. So that is a product of those different perspectives, right.
Steve: So I’m going to just pick on you slightly. I mean you used this phrase which we hear a lot when we talk about research which is like what users want and that’s sort of shorthand, but I think in the examples that you’ve given I think you’re really saying that research is identifying something else. And maybe this is just me being semantically stubborn or something, but no one asked you – and we know this, right – no one said hey I wish I could do this with my phone, right.
Steve: When you say what people want it implies that the researcher is taking requests, but what you’re describing is something much – it’s that hunger and that curiosity and that – and that synthesis to pull those principles out.
Pree: Absolutely, as in we’ve been doing this kind of work for a long time and we always go for these unarticulated needs. What people say, you know, is very easy to get to. And a lot of times what people say is like they’re lying, you know, and they’re saying what you want to hear. It’s getting to the deeper insights by looking at patterns, looking at behavior, looking at what people are making. And so I go back to Liz Sanders who is a mentor from a long time ago. What people say, what people do and what people make. You start to see that each of these areas give you different kinds of data and triangulating between that data takes a lot of synthesis, takes a lot of analysis. And then you get to the core of it and articulate what it is that our users want. And the point that I think is most important and is lost is it is not two researchers going off for two months, coming back with this magic. This analysis doesn’t necessarily happen in a closed room. The right way, the new way to do it is the analysis happens in an open space, happens with others. It is co-created so that at the end of the day our products are – the objectives for our product is co-owned by everybody building the product.
Steve: So let me ask somewhat of a more tactical question about that. You know you’ve talked about some people – some people are paid to have opinions.
Pree: That’s right.
Steve: And so how do you help people be effective in synthesis when what they’re coming in with is what they’ve been working on, what their vision is, what their intuition is, whatever that is. How do you help them sort of embrace the messiness of the data and be in that synthesis process and – I don’t know. I’ll just stop.
Pree: So some of it – so we’ve been doing a lot of what we call iteration zero sprints. We just started doing a lot of those things. Basically it’s design sprints you know. And Google’s done a good job of writing a little bit about the design sprints. You must have seen some of the design sprints work that Google Ventures has put out. A good amount of work which is really good and – and that’s a very tactical way to get a bunch of people together and not necessarily throwing them right into the middle of swimming pool, but letting them wade in the kiddie pool. Now what happens is everybody gets a sense of what the water is like. How cold the temperature is? What it is like to float a little bit? That kind of a thing. The researcher does most of the heavy lifting, or a bunch of researchers do most of the heavy lifting. Then when we all come together towards the patterns everybody is on board. Otherwise you know this is a challenge that our industry has faced for quite awhile, right, which is trying to explain qualitative vs. quantitative research as well, you know. And we do a good amount of both kinds of research within our team and I see a lot of it where – and I’ve seen this in multiple companies where time it’s like 10 users, even though those 10 users were observed for, each of them observed for a few days maybe. Oh, it’s only 10 people. So do you think we can put a million dollars on this or not? And that question always comes up. And so that’s where getting people really connected with the stories, the real people, is super important. Storytelling is a very important part of design and make good design decision. So it comes back to that part of design that we plug into research.
Steve: So how do you answer that question? Are these stories sufficient to commit an investment? What’s the response?
Pree: It usually ends up being yes. So there’s two things that help quite a bit. If you look at venture capital money going into startups, or you look at like companies investing into new products, there is one which is very good data that’s driving certain directions. Second, conviction. When the team sees certain things and they believe in it there is a certain level of conviction. And you cannot necessarily explain it in words, but you can see it. Us as researchers, what we do is we change that part for the whole team. And that’s the value-add that we bring in. It’s quite amazing.
Steve: I feel like yes and no. I mean yes I know what that feeling is like and I know what that conviction feels like and I know when you just – you know that here’s the opportunity because you’ve been out, you’ve been with these people…
Pree: That’s right.
Steve: …you’ve experienced their stories. You’ve breathed the same air as them.
Pree: How do you then translate it to conviction for everybody else?
Pree: You know as in like – as a consultant I’ve done immersion rooms. I’ve done videos, video clips and like it’s not enough. And all of those things help. Now what we’re doing a lot more of is we’re actually taking people into the field. As part of our sprints we have users – we have two sets of interactions. We have interactions in the beginning of the sprint with the users. We have interactions at the end of the sprint with users. And through that interactions, first interaction is about understanding the ecosystem, the domain, the landscape. And the next interaction is about evaluation if you may. It’s more concept evaluation. And so everybody has been part of this journey and it’s not just theater. It is actually immersing yourself in the people’s lives.
Steve: I think you’re describing a really nice version of this – we’ve been headed towards sort of everyone being a researcher, research is integrated, everyone participates. And I think there’s a – we keep talking about tensions. There’s an interesting tension there where this thing works best when everybody plays as deeply and richly as possible and yet it’s not – it’s a profession not a practice. What am I trying to say? There’s a better way to say that. You know filling out expense reports is not a job function at a corporation. It’s an activity that everyone that works for that company does. Research is a job. There’s leaders that have that function. I mean I wonder if this is a point and a transition where – because you’re describing something that’s just really fully integrated at its ultimate version. But they need us to facilitate it, to kind of guide it.
Pree: Yeah, and I’m describing an ideal situation. As in it’s called work for a reason. As in it’s not all like easy and simple. We’re learning and we’re making it happen every day. So it is not – it’s not that everything is like done and like flower petals are put in front of us for us to come there. It is definitely a struggle. At times – like and different teams are different. There are day to day stresses and tensions and those are places where again if we believe in where we want to go as a team I think it makes it much easier because again it comes back to our talk about the local maxima as well. You know even within our profession.
Steve: Could we switch gears a little bit?
Steve: Talk about your choice to join this group and take on this role. You’re describing the reality and the reality is of course everything is a mix of wonderful and challenging and that’s not – I don’t think that’s unique to your situation, but something was compelling to you about this opportunity and can you talk about what you were looking for in…
Pree: Absolutely –So eBay – so my background is engineering and design. Have worked in design research, innovation consulting for a good while. Then moved into Microsoft, was part of the small team that invented Kinect and also was part of the team that did Zune – all that kind of work, like some of the initial Metro designs. So moved to Motorola afterwards to lead the design research team right when Motorola was changing quite a bit. You know Motorola had gone down in terms of how it was doing and there was only one way to go once you go down that much. It’s like up, or disappear. And my personality overall is more – I guess I’m a betting man. I take risks in spaces – calculated risk of course, but I like that excitement. And so I joined. After a few years some of the work was nicely done already. There was things that were in place. So personally I was looking for other possibilities and this thing came along and I again looked at what was happening. eBay as a company is a 20 year old company, has established itself as a leader within of this marketplaces. Makes a lot of money by doing this. However if it continues to do the exact same thing that it did it probably doesn’t have as much of a growth trajectory. And so how – so overall there is a challenge for the company to reinvent, to think about new possibilities. And for an outsider, for me I was like looking at it and I was thinking hey, you know, that sounds like an exciting opportunity. And a place where somebody like me can make impact. The team was amazingly good. The culture was very good. So I felt like hey you know I could work in this culture. And it felt exciting to move at a time when eBay was just celebrating its 20th anniversary, just split from PayPal as eBay by itself and there is a future that we all can create together. Does that answer your question?
Steve: Yes. So if I’m a researcher and I’m looking at different organizations, because you’ve mentioned a number of different orgs that you’ve kind of come into, you know what are things that you would advise me to look for to try to assess – I mean a good…
Pree: A good fit.
Steve: …a good fit for researchers I think is different than other kinds of professions.
Steve: You know what do we look for?
Pree: I think – it’s a very good question and I haven’t thought much about it to be honest. You have researchers at various levels. So there is an entry level researcher who’s done a good amount of academic work let’s say, you know, or has gone through some amount of training. And then there is a researcher with like mid – you know two, three years of experience and then there is the researcher with about ten, fifteen years of experience. Or plus, like much more experience. And then there’s also researchers at various kinds of levels. There are researchers who are IC, individual contributor kind of researchers. And then there are managerial kind of researchers. Different places I think offer different things. So there are certain places – like if you look at the Googles and the Facebooks of the world, if you may, there is a lot of opportunities to learn. And you can have a researcher who is just starting out, a younger researcher who is wanting to learn a lot of things. It’s an awesome opportunity. Even companies like – the bigger companies are really good places for you to learn. eBay is one of those. A new person comes in who has no experience and we’ll train them and they get to observe. They get to be part of a pretty big organization and everybody in the organization comes with a different skill and they all are willing to teach. And they’re willing to learn from this new person as well, you know, and so that automatically makes that kind of an environment better. I would say for a mid-level kind of a person it might be valuable to go to a smaller company, a start-uppy kind of a company where you are probably the only person. You can build out the team. You can figure out what – you know build a network, build a network of mentors, build a network of people that have done this before who can become your personal advisors if you may. And then you can go next step. And as you get more senior there’s only a few places that will hire you as well. Because you’re too expensive for – you know it’s just – or it’s where – or you move into different areas. I know of researchers who moved into product management. I know researchers who moved into design. And you move into lateral areas and grow that way. And all of it is learning and it’s something that as researchers we enjoy.
Steve: So maybe a related question, when you look at people that might join whatever team you’ve been on over the years, what are some kinds of things that you see that kind of – that make your antennae go up. Like what are some good, strong signals that say there’s something here with this person?
Pree: Sure. I think it comes, comes down to passion and you can – again, like when you see people really – and I’d say passion translates into several things. It translates into a real hunger for learning. It translates into caring about the product and the user. And you can see the decisions being made that go towards caring about the product and the user. And I’ve noticed that researchers at a certain point they been taught research so much that they sometimes think of research I’d say in a little bit too pure kind of a way and aren’t thinking about it as a tool, a means to an end. And I see some researchers who have gone past to where they have learnt the tools to cause damage. When I say cause damage as in like they start to think about the product. They start to think about…
Steve: The good kind of damage.
Pree: …the good kind of damage.
Steve: It’s back to your impact point, yes.
Pree: Yeah. So that’s what attracts me to like okay saying hey here’s some people that have potential because they care and they care about the product. They care about the user and they’re very passionate about it.
Steve: That’s great. Alright, so just picking up on this notion of passion, we’ve talked, as of course it’s the focus here, we’ve talked about your work and a little bit about your background and kind of how you got to where you are. I’m wondering about other things that you have passions for that reveal something about how you – what we’ve talked about today? Are there any kind of connections or cross pollinations between research, it makes you do these other things, or these other things kind of impact how you think about research?
Pree: Oh yeah, yeah. No, I’m – I have too many interests as in I’m a Dad. I enjoy being a Dad. I am a bicyclist. I bike. I bike commute to work, plus I bike on the weekends anytime I get. I love cooking. That’s kind of a place where I can calm down. Design, generally speaking like design, designed objects. Well-designed objects, well – you know even furniture making, those kinds of things. I’ve been doing this for years and I do a little bit whenever I get time. Life hacking – it’s just more about productivity for me, as well as just like peace of mind, all of those kinds of things where I could probably get more out of less if you may.
Pree: I also recently got back to this. As a kid I grew up in a – I grew up in a small little town back in India and I come from a farming community and I’ve learned – like a lot of the farming side of things very early in life and one of the things was bee-keeping and I keep bees and that’s something again like you know, backyard honey. I enjoy doing it. It’s a – maybe there is a pattern that goes across all of these too. Have – love old air-cooled Volkswagen Karmann Ghia. Again, the engineering side of it, you know fixing things, making it work again. Those kinds of things excite me. All of it, I think the pattern that goes across is it’s all intense stuff and I like the intensity. Intensity kind of calms me down, and that’s why I end up doing it. And you can see some of these things where I bring them back to work. You know if you think about bees and you think about how it works, the bees as a colony is a brain. A single bee by itself will not sting you if you meet the bee somewhere on the road. You know unless you go and try to squeeze it or something like that. It will never sting you. That’s the last thing it wants to do. But as a colony they work together and one of the ways you can get stung is if you try to steal their honey. And anytime they get a notion that you’re going to steal their honey it’s going to sting you. And they go after it together and there is a whole lot of science there. One bee stings, there’s a little bit of smell that it leaves and then every other bee wants to attack that particular thing. And so now again, simple rules and everybody executes. I bring some of those things back to work. I bring principles and principle-based decision making from cooking. Cooking is an interesting thing. I never cook with a recipe. I cannot cook with a recipe. But I cook with principles. There are certain things that go well together and those have been learned over hundreds and thousands of years. Different cultures have learned this many, many ways. I do read books about food. So principles at work for me come from principles in cooking. Cooking is an interesting thing because if you don’t cook with recipes you kind of borrow principles that help you cook well. It might be about like how you put the right amount of garlic and ginger in your food. Or it might be about like how you pair a certain kind of vegetable with a certain kind of meat. And it’s that connection and it sets – it’s a pattern that works well and if you – you know I read a good amount of books about food. I don’t read about recipes, I just read about food. And I love to read about the history of food as well, as in like how food changed over thousands of years. And those kinds of things intrigue me just because of the fact that there’s this knowledge that has been passed on generation to generation – not necessarily written down, but passed on and it’s a set of principles.
Steve: And so now I’m thinking about what you’re describing and my own inept attempts to cook. If you think about how much garlic or how much ginger to put in you can do it, but if you’re a novice, if you’re like me, the only way you know if you’ve done it right is to taste it, at which point it’s too late.
Pree: That’s right.
Steve: Which I think is similar to some of the work things we’re talking about, like how do you create something and how do you evaluate it. You know you sort of invest – you invest time or building effort…
Pree: You do, right, you do. And that’s also like – here’s an interesting thing as in like from a cooking standpoint. Like my Mom was explaining to me when to add salt to vegetables that were cooking. She had no idea about the science of it, but she was right. You know she was talking about like adding the salt just a little bit after it boils. Right when it boils add some salt and add a spoon of water so that it starts to boil again. That’s what she told me. And I started researching – this is the researcher in me. I started researching why are you saying this? What does this have to do? And then I read up about how the salt added at the right time either makes the vegetable mushy or the skin of the vegetable stays intact, the inside gets cooked. And so it’s a science. And some people have learned it by experience and are passing it on to generations via patterns. And that’s why it’s quite amazing. And architecture is another one like you might have heard of Christopher Alexander and the pattern languages. You know that side of architecture always amazes me. It’s interesting because there is patterns to be seen and patterns that have been happening for years. You know we took the pattern language in the software and a lot of the software now is based on this, you know. So it’s quite amazing.
Steve: You made this point early on about what’s an art vs. what’s a science. Maybe we can take some of what you’re describing with cooking and architecture. Can we talk about just field work? I mean we’ve talked a lot about the context in which research gets applied, but if you think about just doing – field work I think is maybe a rich place to look at just for a minute or so. I don’t know – you know can you draw – I’m going to just throw it back at you. You were sort of talking about cooking as a science but I think there’s an art aspect to that as well, architecture. And so how do some of those principles, do those things apply to field work as an activity in either an art way or a science way?
Pree: Oh absolutely, right. So – and so let’s go to one other passion which is parenting. And I struggled with this, which is being there. Being here now. You know being present. And the mobile phone, the computer, all the screens kind of distract me. And I’m sure a lot of people have the same thing where my kids are wanting my attention and I don’t give them as much attention. Now on the field side of things, right, and this comes to bee keeping. It comes with the mechanical work that I will do on my car. You know all of those things, the attention that you have to give is exactly the attention you have to give while you are interviewing somebody. You have to be there. And you have to feel like that’s the only thing in the world for you at this moment. Is it a skill? Is it learned? I don’t know. But that’s something that everybody – you know – and I know you’ve written a whole book about it. It’s crazy.
Steve: That’s great. That’s a wonderful thought. How old are your children?
Pree: 8 and 9 – you have kids too?
Steve: I do not have kids, no. What should we have talked about that we didn’t get to?
Pree: I think that’s an absolutely good question. That’s a question that everybody should be asking at the end of the interview, yeah.
Steve: That’s right, that’s my stock – it’s one of my stock ending questions – everybody’s stock ending question.
Pree: Let’s see. What should we have talked about? I think we covered a lot of things. I’d say I think we might discuss a little bit and I want to ask you your thoughts on this which is where is design research going long term? And I think a lot about is this field a temporary field that was right for a certain time period and will it just disappear? Or is this a field that will morph into new things. What do you think?
Steve: That’s what I was driving at a little bit when I ineptly compared research to expense reporting. I mean I’ve thought for years that there’s a potential for this to be an activity not a role and it’s interesting – I don’t know, it’s very exciting to hear how the activity of research has really, as you’re describing it in these ideal situations, it’s really grown. Like what we mean by research and what it entails and the skills and all that. It’s – there is a point, and you probably remember this from your early days, where, you know buying a video camera was like, that was sort of the step into doing research. Well we bought a video camera. And it was sort of a naïve time, but I love hearing you describe kind of the – I guess the maturity and sophistication and the capability that this set of activities has. But I have just been saying for years like is this just design? Or should it just be design? And I feel like – I don’t know if this is the same trend or an adjacent one. Part of that sort of questing that you and I and people in our industry have been doing, sort of evangelizing, advocating, hoping, wishing, wanting, has been to see research more fully integrated, more fully adopted and I think with that has come a couple of things. There’s sort of a good and bad and as you said there’s an ideal and you’ve laid out the vision for the ideal and you strive every day to achieve that ideal. I also think, and hopefully I’m not just being grumpy about it, but there’s a lot of commoditization of research that’s happened at the same time.
Pree: Oh absolutely.
Steve: Where there’s lots of junior people coming in. They’re not always getting the mentorship that you’re describing. They’re being asked to kind of execute closed end – not explorations. They’re barely doing evaluative. They’re swimming in the local maxima to mix metaphors. And that some people are kind of you know dusting their hands off going well we did it. We hired a researcher, we’ve got research being done. So…
Pree: It’s a checkbox, yeah.
Steve: Yes, yes. I guess that isn’t new. I guess I just see more versions of that where there was a time where research was ignored and now it’s sort of pulled in, kind of kept in the corner and not doing very much. And I feel like oh we wanted everyone to do research and now they are, except they’re not always doing – they’re not always kind of doing it as a leadership, strategic activity the way you guys are and so many of our peers are and so sometimes I wonder like well is this the consequence – is this the downside of us wanting this so much that it has been adopted but not in the way that we had hoped for.
Pree: Yeah. And I think you mentioned something very interesting which is this is an activity that everybody does or it’s just one of those activities. It’s not necessarily a discipline. It’s not a profession. It’s a very interesting thought because I have a sense that you know over the last few years, like it’s been 20-25 years that we have had kind of a new discipline come about which is the product management discipline. You know it used to be more program management before or it used to be more like putting – connecting all the dots, rather than owning the product. And I also wonder out aloud if we somehow connect with the product management discipline somehow or the design discipline even more so. And I don’t have any conclusions. It’s one of those things that kind of keeps me up and I think about like where is this going?
Steve: I’ve said this before on this podcast, it seems like research follows design or follows user experience and that I think some of the things we wring our hands about are things that that profession, that community, was wringing its hands about three years ago. And so the move – I hear UX people say all the time that product management is kind of, is the opportunity for UX to grow. Rosenfeld Media did a virtual conference recently that was like a PM plus UX event. So there’s these signals that these are the conversations that are happening. And so where’s the research side of – ‘cuz sometimes research is sort of like hidden under UX a little bit.
Pree: It is, but if you see some of the latest things that are happening, I feel like research is actually leading design and the reason I feel research is leading design is because of a few things. One is how connected we are with users? And how that gap between a user having a need to a solution that can be shipped is kind of reducing. As that happens you end up having a lot more of this need gathering. A lot more of this co-creation if you may of the end product. That is important. And we’ve always been at the leading edge of this. We’ve been pushing design from design being just inspiration and like let me make this up, to design that is like intentional. And we continue to push this discipline. So I would any day call it as leading the discipline rather than kind of following it. Following it maybe from a time standpoint, maybe. Yes I see where you are going with this, but yeah.
Steve: So that dynamic is interesting. And maybe design is maturing to not be decoration. I mean it’s been maturing that way for a long time and that’s – I’m just reflecting what you’re saying – that starts – as it matures to be sort of creating experiences, creating value, then the hook for design to do that is research which then goes back to your question, which we’re not going to be able to answer, which is should it just go away or will it just go away?
Pree: I don’t know. Like – another interesting development that’s happening with the design side of things is patterns. So over and over again now we are starting to see design patterns that are used multiple places. And so suddenly the value of having to design individual things go down. And it happened in software a few years ago. As in some of the very good software programmers know that the best thing to do is to have a collection of all of these things that they use over and over again and they just pull it in, cut and paste. And most of the code is not written. It’s just cut and pasted together. I’m not reducing the value of a person – of software here. You know I can code and I know it’s a lot of work – patterns and knowing when to pull those patterns together is the skill. So you mature the discipline to something else. And we are very good as a research discipline to figure out those kinds of things, the needs or what patterns to pull together. So I think we’re set for good success. That’s what I’m thinking.
Steve: I love that. Maybe we should leave it on that wonderful high note and thinking about the success we’ll continue to have as a discipline. So thanks very much, Pree. It was a wonderful conversation.
Pree: Absolutely. I enjoyed it. Thank you so much.
Steve: Thank you.