Gustaf Alstromer, YC’s newest Partner (formerly product lead for Growth at Airbnb) joined a panel with Ed Baker, (former Head of Growth at Uber), to share tips on growth experiments and team dynamics at a scaling company.

Topics discussed:

What is a north star metric for growth? If you talk to anyone on the growth team, and ask them, “what number are we trying to grow.” They’d be able to say that number. And if they aren’t working on something that could grow that number, they’re probably working on the wrong thing. At the time Ed was at Facebook, retention and user engagement contributed even more to the north star metric (MAUs) than new user acquisition.

Paid acquisition can further accelerate growth: Growth isn’t only about organic growth — especially for companies who charge for their products. In a competitive market, it can make sense to pay to acquire up to the potential value of a customer if it’s important to grow quickly (this was true for Uber, who had to beat competition to market).

Is there a channel with particular upside right now? Nothing beats building a great product. Beyond that, most great companies get really good at one specific channel — so figure out what your product is best suited for and double down on it (make sure it’s a channel that can scale — meaning that channel potentially has 100s of millions of people coming through it).

You should be seeing things that are counterintuitive in your data: If you aren’t, you probably aren’t experimenting enough.

A little wording change can matter: As an example, Facebook saw low growth in Japan, and went and talked to people in the local market and found that people felt it was rude to “invite” friends to Facebook. They found through a growth experiment that changing the wording to “announce to your friends you’re on Facebook” (instead of “invite”), they immediately began to see growth there.

Make it easy for users: The simplest thing you could do today to help strengthen user data is to not log people out. This practice is bad for growth, and Amazon is a great example of doing it right.

Eventually, the whole company is a growth team: At Airbnb today, every big team uses data to fuel decisions. That’s the goal. The growth team should not only be collaborating across the business, but you should start to see most areas join in on a cadence of experimentation.

This panel was recorded at The Scaleup Offsite, an event for growth-stage CEOs hosted by Y Combinator Continuity and Greylock Partners. Josh Elman from Greylock Partners moderated this panel.

---

---

Subscribe

iTunes
Google Play
Stitcher
SoundCloud
RSS

---

Transcript

Craig Cannon [00:00] – Hey, this is Craig Cannon and you’re listening to Y Combinator’s podcast. Today’s episode is about scaling growth. It’s a conversation between Gustaf Alstromer, Ed Baker, and Josh Elman. Gustaf is a YC partner and prior to that he worked on growth at Airbnb, Ed worked on growth at Uber and Facebook, and Josh is a partner at Greylock. The conversation was recorded at The Scaleup Offsite, a private event hosted by Y Combinator Continuity and Greylock Partners in April 2017. Okay, here we go.

Josh Elman [00:27] – What’s pretty cool is we have two guys who have been living the center of building up these growth teams kind of for the past seven or eight years. Ed joined Uber to start the growth team when it was five people and then over the 3 1/2 years became VP of both product and growth for all of Uber. And Gustaf joined and the Airbnb growth team was him and two other people when he got started a few years ago and that has scaled to over 100. So it’s fun at Scaleup Offsite to talk about scaling. We’ve mostly been talking about scaling organizations. This is a chance to talk about how do you scale the organization that continues to scale your users? What’s fun for me is I’ve known Ed and Gustaf now for most of a decade in this growth world and we were all started back when this idea of a growth team, this idea of a team that’s focused on this intersection, product and marketing to build stuff was pretty novel and almost didn’t even really exist. And we were all doing it in the social world and I remember Ed started a company in 2008 called Friend.ly or as… and then Friend.ly. You know Gustaf started a company in 2007 called Heysan and then went over to Voxer. And these were like, how did we get to millions of users in the early days? So I’m curious if you could just talk about this heyday of social, viral growth explosion. And what are some of the key crazy things you learned then that have really helped you as you think about growth today? Ed, I don’t know if you want to start.

Ed Baker [01:51] – Sure, so yeah I guess 2007, summer of 2007 when Facebook launched the platform was I think part of that explosion of just crazy viral growth, spammy apps on the Facebook platform. That was actually the first, those were the first apps I built that reached tens of millions of users within months’ time. And I think it was a great platform on which to just experiment with a lot of different ideas because it was so easy to build these simple Facebook apps. And just run lots of A/B tests. And so much of figuring out how to grow something is, it’s a combination of an art and a science and by running lots of experiments, you start to build an intuition for the types of things that work and don’t work.

Josh Elman [02:39] – That’s great. What were some of the early tricks you found that made you go super viral? You were pretty famous back in those days.

Ed Baker [02:44] – Yeah, I mean you were at Facebook back then so I think Facebook created new rules after I found some loopholes.

Josh Elman [02:54] – Yeah, almost often yes.

Ed Baker [02:56] – There was one I remember, I think the most viral Facebook app, I’m hesitant to even admit this, but it was called Zodiac Photo Album and the way it worked was, literally with one click it would create a photo album with 12 photos in it, one for each zodiac sign. And then show all of your friends who had that zodiac sign in each of the 12 photos and tag all of them. And I think we were allowed to tag like 20 people per photo. So it would show up to 20 of your friends per zodiac sign times 12. So with one click, you’d tag 240 friends. And all 240 would get an email saying, “Ed just tagged you in a photo.” So, that has a very high response rate. People kind of want to see when they’re tagged in a photo. They would go, they’d see oh, it’s a zodiac photo album, create your own with the Zodiac app. Within, I think within 24 hours we had tagged tens of millions of people in photos.

Josh Elman [03:56] – Incredible.

Ed Baker [03:57] – I think including Zuck and then I heard the platform team shut it down shortly after that.

Josh Elman [04:02] – I wonder if those days are over of those crazy viral pops. And then Gustaf, you were really central to a lot of the early experimentation on mobile. I remember we first met when Voxer was blowing up. And you guys had figured out some really unique ways to blow up using notifications and inviting. Can you remind us, how did you get to this world of growth and how did you figure out stuff in those early days, especially on mobile, which people think is a lot harder even than social was?

Gustaf Alströmer [04:29] – So before Voxer, I was at a company called Heysan where I cofounded and it was a product, a messaging product on mobile that we couldn’t grow so we spent a lot of time trying to learn how to grow. When I joined Voxer a couple years later, the first approach and first thing I kept in mind is we need to think about growth. And at the time, back to the early question, I think there’s a couple of growth becomes important and when they have new platforms. So you have Facebook platform, you have the mobile platform, you have Google, SEO, there’s a bunch of larger platforms that makes people actually care about investing in them. At Voxer, the iOS platform was key to those things for the address book and push notifications. Those are things that weren’t there in the very beginning and once they started happening, people started exploring what you could do with those things. With Voxer, when I joined the most important thing… So Voxer was a walkie-talkie app on iOS and Android. You push a button, you start speaking in the other end. We found that the amount of friends that you have when you open the app for the very first time on the app and how you display them was absolutely key to whether they were going to use the app or not. So we spent basically the first year optimizing those things. Like basically, how many people do you have when you open up the app for the first time? And how do you get to a list like that right when you open the app? Well, you match phone numbers with address books. Spent a lot time investing in matching phone numbers with address books, not that different from Facebook platform. And we tried to optimize how many address books and how many phone numbers we got from people. And we’re doing this in the best interest of the users. You wouldn’t end up using the app unless you actually saw friends on the app, so that was basically the goal. This was all in the best interests of the users and sometimes you forget why you do things.

Josh Elman [06:16] – So Ed, I remember your company got acquired by Facebook. And you jumped into a little bit more a mature growth team at Facebook. You’d been kind of virally hacking and building these things on the platform. What was the adjustment to go from being in a sort of experimentation mode to something that was a little bit more mature? But what did you see when you observed that mature Facebook growth team?

Ed Baker [06:37] – Well one thing I saw was I was finally working at a company that had an amazing product that people loved. And I had just never actually built that myself before so that was pretty cool. And with that came amazing retention, amazing engagement, and Facebook was already at a size where retention and engagement were bigger drivers of the North Star metric MAUs than new user acquisition. So everything I had worked on in the past was all about how do you get those first million users? At Facebook, it was how do we keep the users we’ve already acquired, keep them engaged, minimize churn? And to me that was one of the biggest differences and most of the growth team at Facebook at the time was actually focused on friending because we discovered at Facebook that friending was the number one driver of retention and engagement. So then there were teams on the Facebook growth team that were basically focused on things like PYMK, people you may know and other ways of causing people to form more connections. So I think that was one big takeaway. The other one, we can talk about it later is just the way the team was structured. I know you got into that a little bit, their different functions as the team grows.

Josh Elman [08:02] – Actually, can you walk us through that? Now’s actually a good time to think about how was that structured? Because then I want to get into this transition to these different real-world companies.

Ed Baker [08:11] – Sure, sure, the real-world companies yeah. Well so, one thing I learned at Facebook was, I really liked how the growth team there was structured with all functions. So product, engineering, analytics, design, and even marketing. So those five functions were all critical to work together and have all within that growth org. And when I was at Facebook, I think the growth team was around 500 people or so, but across all those different functions. In fact, all of analytics back then was part of the growth team. And I took a lot of those learnings when I went over to Uber and started our growth team there with just initially a five person team. Kind of picked one person for each function. But yeah, I think that was one of the big takeaways from Facebook.

Josh Elman [09:13] – And you know that’s a perfect transition because when I’ve been talking to a lot of people about what the real success of Uber and Airbnb. I mean, Joe gave a great talk about why Airbnb has become so important in the world, but a lot of people have said it’s kind of, we took these hotels and transportation, which were old world, real businesses, and applied a lot of these digital growth technology and learning to them. And so, talk a little bit about what it was like going into Airbnb and starting the growth team with two people, and Uber and starting the little growth team. What did you see there and how did you start this sort of practice from what you’d learned before?

Gustaf Alströmer [09:50] – Sure, so I think the… I can’t speak for Ed, but I think the initial idea of a growth team is that there is some function of the company that really cares about metrics, they’re really optimized for the goals. And some time ago this was unusual in startups. Most people did not look at metrics when they made product decisions, they did not experiment. So instead of making everybody responsible for that, you made one team responsible for that. And that ended up being the growth team. I think really great companies today, they have everyone caring about metrics, using experimentation, and doing all those things that growth teams do to make decisions. So I think things have change a little bit and the role of the growth team as the team that evangelizes those things have changed a little bit. When I joined Airbnb, a lot of people joined at the same time. They had experience from all those things. So pretty quickly I got a lot better set of tools than I had ever had at Voxer, around experimentation, around support from data science. And that just, I would say in some sense made all the infrastructure of doing a growth team at Airbnb really, really easy compared to my previous experience. And I learned a ton from all those people and it’s been pretty incredible. But yeah, in the very beginning we, like everyone else had, Airbnb had always been an incredible product and as a company we along those years have learned to identify and optimize every single step of that experience. A lot of it is of course offline, but of the stuff that we do online there’s a lot of things that we can do to just measure and make the user experience better. And that’s what we’ve been doing. But I think yeah, the role of the growth team have changed from being that evangelizing team that do all of it to evangelizing to the rest of the company so that everyone can do product in a very similar way.

Ed Baker [11:44] – I’d say at Uber there were three big differences from Facebook when I went over there, in terms of forming our initial growth teams. So the first was, it’s a two-sided marketplace and when we came up with our North Star metric, I remember having conversations with that small team of five, one of whom Mike Pao is sitting in the audience here.

Josh Elman [12:04] – Actually, can you define North Star metric? You’ve used that a couple times. I think that’s actually really important for the growth discussion.

Ed Baker [12:09] – Definitely. So at Facebook it was monthly active users. It’s basically the one metric that, the way I like to think about it is, if I talk to anyone on the growth team and ask them what number are we trying to grow, they would be able to say it’s that number. And if they’re not working on something that ultimately is growing that number they’re probably working on the wrong thing. So that’s kind of how I like to think about it and it’s helpful to set goals. We’d set every half, we’d have this is our six month goal, but then we’d track our progress every week and see are we on track, and there’s of course seasonality and stuff like that. It’s not just a straight line, but you could kind of see how are we doing versus our goal. So at Uber when we came up with our North Star metric, we had to consider there are two sides to the marketplace. So it’s not just monthly active riders. Ultimately we decided on trips, weekly trips as the North Star metric because for every trip to take place you need riders and you need drivers. And so, by having trips as that North Star metric, you could then break that down into the supply side and the demand side. And you need more riders, more drivers, and you can do growth accounting on each side of the marketplace to say, well to get more riders, you need more new riders, reengaged churned riders, and minimized churn of your existing riders. And same thing on the driver’s side. So, that was one thing was agreeing on, this is our North Star metric in a two-sided marketplace. So that was a bit different from Facebook. I’d say a second thing was because Uber is a real-world company, there were several parts of that funnel that you can’t just measure online. Whereas on Facebook you can track every click, every pixel, all of that stuff. With Uber, there are steps of that funnel where it involves a driver going in and getting their vehicle inspected or uploading their documents and ultimately downloading their iPhone app, going into their car, and doing that first trip. And for me it was super eye-opening when I did my first trip as a driver. I walked out to my car. I’d done everything, I was activated, I walked out to my car, I walked back to my house because I got so nervous about doing that first trip. And that’s when it suddenly hit me, this is why so many of our drivers who are activated never do that first trip. So that real-world component added some extra challenges in terms of one, how do you measure it and two, what are some things you can do to influence the way people behave in the real world. And then, let’s see. I said three things and now I’ve mentioned two. So, two-sided marketplace, the real-world component to it, and I’ll come back to the third when I remember it.

Josh Elman [15:09] – Your different North Star metric. Well and so, as these teams have scaled way up, you said, talk about data analytics, design, product, sort of internet marketing.

Ed Baker [15:20] – Oh that was the third by the way. Paid acquisition.

Josh Elman [15:24] – That’s what I was actually ask for is…

Ed Baker [15:26] – So Facebook, we didn’t really do very much paid acquisition, but at Uber we had an entire performance marketing team. Well initially it was a one-person team, but ultimately became a team where we would actually spend money to acquire both riders and drivers.

Josh Elman [15:41] – In our social worlds, we’re all about these organic, how many people can we organically get into our products? And if we spend a dollar on user acquisition, we always consider that a dollar too much. And you know the boards are like, “How do you get more free virality?” But I think both Uber and Airbnb because you’re businesses people actually pay for, how do you think about paid acquisition differently than this sort of organic acquisition? How do you train a team to understand and think about that differently?

Ed Baker [16:10] – Do you want to start or would you…

Gustaf Alströmer [16:11] – Sure. So let’s see. I would say a lot of those platforms that you are doing growth on, Google, Facebook. They are turning, the opportunity of doing free growth there is getting smaller and the opportunity of doing paid growth better is getting larger, so most good growth teams should invest in paid marketing. I think that, in my experience of looking at other companies, what they’ve been doing, large companies that do paid marketing, they tend to integrate that with the product team and engineering team. So there are engineers and project managers supporting those teams. They call it the ad tech or something like that. And I’m sure both Uber and Airbnb is at that stage right now where basically, most of the leverage you’ll have at scale is going to be through engineering, data science, and product management. That’s a different world than when you start off. When you start off you have a smaller team. And yeah, that’s probably it.

Ed Baker [17:11] – Yeah I mean, I’d say the way I think about paid acquisition is it can just further accelerate growth. So, Uber would have kept growing even if we had never done any paid acquisition, but especially in a competitive market it makes sense to pay up to what that rider or that driver is ultimately going to be worth. When you’re in a situation where the faster you grow the better because as you get bigger and bigger there are a lot of other efficiencies that kick in. So the whole network becomes more efficient once you have more liquidity. So there’s value to getting there as quickly as possible. It’s kind of tricky to know exactly how to calculate the LTV of a rider or a driver in a two-sided marketplace because the riders pay the drivers and some of that money goes to Uber. So how much is a rider worth versus a driver worth? And we had a team actually focused on that exact question, team of data scientists. And it was more complicated than I understood, but basically there’s certain markets that are more supply constrained where the incremental driver’s worth a lot more, other markets that a certain time of year might be more demand constrained where you might be willing to pay more for the rider and that varies over time and by geography.

Gustaf Alströmer [18:31] – So a quick comment on that, I would say most companies that grow really, really large, they do it on figuring out one platform. So they are really good at online marketing, they’re really good at SEO, they’re good at virality or the Facebook platform or something like that. It’s rare that you have, like I figured out all the platforms. So, I would think about the product that someone have, the product that you have and which of these platforms is it the most suited to. Is it most suited to search, is it most suited to display, is this most suited to virality. It really depends on the product I would say.

Josh Elman [19:01] – Interesting, and where do you guys see any upside right now in the marketing or growth world. It seems like a lot of these channels have gotten pretty saturated. Facebook’s gotten relatively expensive to be buying on. Google’s various platforms and video, you know I’ve heard a lot of people talking about video ads. They’re sort of the new place. I see people are even going and advertising on TV. Where do you guys think about where those growth opportunities sit out there now?

Ed Baker [19:31] – I mean, I come back to build a great product. Because if you have that product market fit and the people using your product love it, all of the channels are out there today that will allow people who love a product to share it with their friends. So I think that’s actually the most important thing. And then everything else will follow.

Gustaf Alströmer [19:50] – Yeah, I don’t think there’s anything short term that will remain, like successful for a very long time so the best advice is invest for the longterm, whether it’s SEO, whether it’s paid marketing, whether it’s virality, take a couple of years, look out and see where do we want to be. And aim for, look for channels that actually do have true scale. Like channels that have hundreds of millions of people discovering products through those channels. And then invest for the longterm. And engineering and data science is critical to be successful in my opinion.

Josh Elman [20:25] – I’m surprised neither of you mentioned Instagram, the tip I got yesterday at F8, talking to a bunch of Facebook folks is they think Instagram app install ads are significantly under priced relative to Facebook. So if you guys go back and start buying on Instagram, you might be surprised at how it does. So, one of the unique things again, in the social viral world, we grew, scaled on the internet. You guys also have to grow city by city. What are some of the things that you did to set up teams for growth that could actually do growth in a city that could customize it for the city or how much did you try to make it a centralized playbook that you’d control from the mothership?

Ed Baker [21:00] – So, every city would have its own city team, beginning with a launcher. Again I mentioned Mike in the audience. He was a launcher, then was our general manager in Boston, and then basically headed up product for the growth team so it was really valuable to have the person who was running product for growth be one of these ex-operations people. And working with all the city teams was really important in terms of just making sure that product and operations were integrated. We now have a team at Uber called Product Ops and it’s basically a team that is that bridge between the product team and the operations team. But as much as possible we tried to just have a playbook. One thing that’s helpful with Uber being a global business, and I’m sure you guys see this as much, if not more so at Airbnb, is when we would launch in a new city there would already be a lot of pent up demand. We’d see a lot of people trying to use Uber in a city before we’d launched in that city. So there were certain things we could do to kick off that flywheel. But I’ll share one example of something where we actually did something very different to the product in another city. And this was back when we were in China. We found that it was hard for drivers to sign up to become Uber drivers in China because of the firewall, there were latency issues, and signing up online. And a lot of drivers had phones, but didn’t have computers and it was just hard for them to sign up. So one of the engineers on our China growth team built a WeChat bot. WeChat’s a messaging app, or the Facebook and also the internet of China really. And so we built this WeChat bot where as a driver you could take a picture of a QR code and then it would just start asking you questions. What’s your name? What’s your birth date? What’s your driver’s license number? And people would quickly be able to sign up to become a driver. And we’d have these onboarding sessions with hundreds of drivers, if not even more than that, showing up in one room, all on WeChat signing up, which was amazing until WeChat started blocking us because they were owned by Tencent, which had invested in our competitor. So that’s one example of something we had to do very differently in a different market.

Gustaf Alströmer [23:25] – I would say there’s two things I would call out. One is translation, so absolutely key to have a way that you can translate content across languages. Often overlooked and it’s a huge growth driver. A lot of people don’t speak English and need the experience that you’re offering in a different language. The second thing is what I call product gaps. So if you look at your product globally, there are some parts of your product that won’t work in some countries. Whether it’s authentication like Facebook Connect, whether it’s payment mechanisms, search, SEO, and just identify those gaps. There are not that many countries in my experience that are dramatically different. There are maybe five or 10 that are quite different than US, but most countries are not that different.

Josh Elman [24:07] – Cool. And just last thing because growth is all about experimentation and often experimentation against instincts. What is one great case where you and your growth team really either believed something was going to make a huge impact and then were disappointed when it didn’t, or where someone just did an experiment and even you, you were like this is never going to work and then it blew you away?

Gustaf Alströmer [24:30] – So we have actually institutionalized this thing. We call it the Experiment Review. Every two weeks the growth team gets into a room and we show experiments that we’ve shipped and gotten significant results on in the last month say. And the engineer will go on stage, show the experiment, show the control and the experiment. Before they show the answer, they’ll ask the audience, “What do you guys think? How many thinks control, how many thinks experiment?” And it turns out that we often disagree. Most of the times we disagree, which is exactly the point you’re trying to make by doing experimentation. That predicting product decisions is really tricky. So I would say this happens all the time and if it doesn’t happen then you’re not taking big enough risks. You should be seeing things that are counterintuitive, sometimes when they’re very counterintuitive having experience of seeing a lot of experimentations makes you ask the right questions. Oh maybe the metrics is wrong this time. That happens as well. But yeah, things should be really counterintuitive. I’ll give one tidbit and one advice. If you have a website and you have people coming to your website and you have accounts, don’t log them out. Keep people logged in for as long as you can. There’s really no benefit in automatically logging people out after a couple days in terms of growth. So I would look for Amazon for some advice on how they’ve done things.

Ed Baker [25:49] – So one thing I found interesting is often it’s the smallest tweaks that can make the biggest impact. And there’s not necessarily correlation between how much work you put in and how much of a step change it makes or sometimes an inverse correlation. So I’ll share one interesting example from Facebook. We were looking at the sign up, or basically the funnel and the viral metrics by region and we noticed that Japan was our least viral country. And when we dug in and tried to understand why, we saw that very few new users were actually sending invites to any of their friends. And we had a small team in Japan. I spoke with the country manager there and he said, “You know people just think it’s rude to send invites.” It’s like a cultural thing, but people don’t want to intrude on their friends and send invites. So we brainstormed a bit and we decided to literally make a copy tweak. And instead of calling them invites, we called them announcements. And we would say, as part of the sign up flow you’d import your address book to find your friends. And then instead of saying invite all your friends, we said, “Let Facebook now announce to all of your friends that you’re on Facebook and this is where they can find you.” And Japan went from our least viral country to our most viral country. And when Zuck saw that he was excited and thought why don’t we do this in some other places as well. But it was literally just a copy tweak to make people more comfortable with sharing the product they were using with their friends.

Josh Elman [27:31] – That’s great, I’ve been talking to some mobile developers that even the different of invite versus add somebody, which is vaguer. If you change your language from invite to add, people will send the SMSs out to their friends to invite them to a service at a much higher rate than the word invite so these really subtle things they don’t expect. We have time for one or two questions from the audience. Anybody have any? There.

Audience [27:58] – Ed, you talked about the North Star and you said at Uber it was about the number of rides, but you could have easily been optimizing for crazy retention and zero growth, and new user growth or new rider growth, or all new rider growth and to news last comment, horrible retention. So how did you balance out knowing what your almost sub-North Stars were so that you could have the best North Star if that makes sense.

Ed Baker [28:22] – Totally, yeah that makes a lot of sense. And it actually changed over time. So early on, we were kind of in land grab mode where it was all about new user acquisition, new riders, new drivers, but we would look at growth accounting. So you’d say well, the total number of active riders or active drivers equals last week’s active number plus any new users you get minus any users that churned plus any users reengaged. And you can look at each of those three things, kind of the magnitude of each. And early on in the growth curve when you’re at the bottom of that S-curve, the new user acquisition number is probably going to be much larger than the churn or the re-engagement. And so it makes sense to focus the most on growing that number. But as you get bigger and bigger, as your active user base gets larger, the churn as an absolute number goes up because it tends to remain a constant percentage or roughly constant percentage of the absolute number. And then at the same time, new user acquisition gets more difficult because you’ve already reached out to so many new people. So if you just look at the growth accounting and see those are the levers, which ones can have the biggest impact. Early on it might be new user acquisition, but later on it’s probably going to be retention and re-engagement. But that said, you kind of need that retention from the beginning because if you’re just, I’ve grown plenty of things in the past that haven’t had retention and it’s fun while it’s going like this, but if that retention’s not there ultimately it’s not really going to help.

Josh Elman [30:09] – Is that like zodiac calendars?

Ed Baker [30:10] – Exactly, that kind of stuff.

Audience [30:12] – To Ed, earlier you told a story of the challenge of getting people to go from registering as a driver to actually overcoming the nervousness to go out and do the first trip. As a driver, I’m interested in what are some of the experiments you ran there and what you learned about how to actually get people to do that in the real world.

Ed Baker [30:31] – Yeah, definitely. Definitely, so I’ll just mention one example is we saw that sometimes a new driver would log in and might not get a trip request right away. And sometimes that driver might just be waiting for a really long time because they’re a brand new driver and maybe they’re waiting somewhere, maybe they are waiting in front of their house and there just isn’t a bunch of stuff happening right there. Or there’s another driver who’s more experienced and who knows if you wait a block away from here, they’re more likely to get a request. So one of the experiments we ran was what if we make sure that that first experience is better. Are there things we can do to get that driver hooked? So that’s one example. We also experimented with, are there ways we can get potentially existing drivers or other people to get on the phone with a driver and talk them through that first trip. And so, the team’s experimented with a bunch of different things like that.

Audience [31:45] – Just to kind of probe on something Gustaf alluded to regarding growth organizationally. Do you guys think that the right steady state outcome is for all teams to become growth-oriented and have growth people inside of them or to have a separate stand-alone team that’s kind of fully vertically integrated? And if the latter, what conflicts have you seen between marketing, who’s focused on owning the homepage and the language and the brand goals, or lifecycle who thinks that they own retention versus growth, and likewise with product groups that feel like they own a product and customer experience that they’re trying to get with you guys kind of hacking away at that?

Gustaf Alströmer [32:23] – So, I would say that ultimately the company becomes a growth team in the sense that you use data to make your decisions. You basically look at the data because it gets so hard to predict what are the right product decisions to make. Once you make optimizations. So ultimately at Airbnb, all the larger teams are using data to develop the experiments. And that’s kind of, the idea of a growth team for us means where you sit in the funnel. You’re at the top of the funnel, you’re finding users that are not already using Airbnb. In other companies, growth teams is a smaller organization that kind of evangelizes that to the rest of the product team. But Airbnb, really the entire product team is now a growth team and the goal of the data team that have built these tools is to get everybody to use them. They’re literally measured by how many people using experimentation frameworks.

Ed Baker [33:16] – We actually merged the product team and the growth team about a year ago at Uber largely for those reasons, especially as we started focusing more on engagement and retention within the growth team. And that’s kind of what the core product team was working on as well. It just made more sense to bring it all together.

Josh Elman [33:36] – And I would just add, as you talk about your lifecycle, and your homepage copy teams and everything, and brand that actually when they kind of merge together and they have the orientation where growth and marketing are one thing, as opposed to two, you understand everything we do is in the spirit of growth, even if it is driving our brand or other stuff, rather than trying to keep them separate and contentious is really that merging. We have time for one more question.

Audience [34:04] – Question for Ed. Uber was super aggressive in churning, growing. Were there any really crazy things you guys discovered or hacks there that maybe people could use back here?

Ed Baker [34:15] – I’m sure there were, but I probably can’t talk about them. I mean actually, I don’t have any super crazy ones that are coming to mind right now, but I probably shouldn’t talk about any of the stuff that worked really well just given that we’re probably still doing it if it did. Sorry about that.

Josh Elman [34:36] – Awesome, well thank you. Hopefully you’ll be scaling your orgs and your user bases together. Thanks, Ed and Gustaf, a lot for this.

Craig Cannon [34:46] – Alright thanks for listening. So if you want to read the transcript or watch the video, you can check out blog.ycombinator.com and as always, please remember to subscribe and rate the show. See you next time.