Ep. #9, Demystifying Growth Hacking

Craig Kerstiens: So welcome back to another episode of Practical Product. I'm Craig.

Rimas Silkaitis: I'm Rimas.

Craig: And today we're here with Sean Ellis who actually coined the term, Growth Hacking. We can all thank him for all the opinions that we have about growth hacking. Let's just kind of jump in right there. Can you summarize it in two sentences, is that possible? I think we are going to spend maybe 30 minutes on the subjects. But what is grow hacking? Everyone's heard the term. You coined it. What would you say it is?

Sean Ellis: And there are a bunch of definitions out there so I'm glad you asked that. It is not just creative marketing which is what a lot of people kind of think of growth hacking as. It's a little more boring than that.

Growth hacking is a process of experimentation across all of the levels of growth, going across the full customer journey from consideration of a product to becoming an active, valuable user.

Somebody who's talking about the product. All the experimentation you can do across that journey to improve growth.

Craig: So I hope we didn't put any people to sleep. That actually sounds like real product work there, not like a magic bullet.

Sean: No, that's the biggest confusion. It attracts a lot of people when you say here are 22 magic bullets guaranteed to grow your business. People want to listen to it but unfortunately, most of the time those don't work and if they do it's fleeting growth that disappears really fast.

Craig: Yeah and I think there are great exceptions that we've seen. There's Dropbox and I think Uber, there's some good examples of a few tricks that have worked really well. But there is also the question of how do you do this very sustainably, repeatedly?

Sean: And having been the first marketer of Dropbox, I can tell you that at the source of all those tricks was a super valuable product experience that if people didn't stick around and stay retained on that product experience, they would not have grown to the size they are today.

Rimas: Interesting, let's dive into this. What I'm hearing is that it's really about retention. Is that what we're talking about at the heart of all of this?

Sean: It's about retained growth so retention is a critical part of it. But it's about understanding the value that a product delivers and instead of trying to grow registrations, and trying to grow some kind of number that doesn't necessarily reflect that value. It's about getting people to an experience with a product where they say,"Oh my God. I can't live without this." And optimizing all of your efforts, to get as many people to that experience as possible and that's how you grow a business. It's not just about sort of having them hit a website.

Rimas: Warning to all of the listeners out there. You better have a good product first before you start doing any of these other stuff is that correct?

Sean: It is but it's kind of interesting because people say well then you're really just relying on a good product and a good product is not just something that happens. A good product is also about getting the right people to use the product in the right way and that's a lot of what growth is.

Find your most passionate users first, learn everything you can about them. And then figure out how to get more people who have the same needs to the same experience in the product, and set the right expectations, and reduce the friction of getting the right experience.

Craig: I think that sums up how I think about product management as well. I think Marty K talked about product management is making sure you bought the right product. Engineering is making sure it's build the right way and that's kind of a cohesive cycle. Okay, so we've dispelled some of the ideas already really quickly about grow hacking.

Sean: Yeah.

Craig: This actually sounds more interesting from a product perspective. I want to jump in and say retained growth, that sounds super sustainable for my business. This is interesting. There's a bunch of facets to it and I'm going to jump in and I'm just going to post a listing for a growth hacker as my next step.

Sean: Not necessarily so I think one if you post a listing for a grow hacker. You're probably going to get the wrong type of person applying because they're probably the magic tricks guy. Ultimately it depends on the size of the company. If you're super early stage company then maybe you do want someone who's super creative and who's going to help kick start that growth. But they should at least be thinking about it holistically and getting people to a valuable experience.

Craig: What is super early stage?

Sean: When I was at Dropbox, there were seven people. That was super early stage.

Craig: That is pretty super early. I think we have a lot of listeners. We have something like that 5, 7 stage. We had some at 22, 25. We have people 100 person company so I'm sure it's larger but a full spectrum there.

Sean: Right and then as you get bigger. As you get to even 50 people or 100 people then it's much more of a team effort. It's this cross functional effort. Traditional organizations are basically built with silos. It starts with the marketing team. Maybe interacts with the sales team a little bit and the product team gets in there.

Might be some customer support or success along the way, and it's really hard to co-ordinate that journey and there's a lot competing metrics along the way. What growth teams have done is that they're really organized across that full customer journey. They're cross functional. They have a North Star metric that really defines aggregate value delivered to customers and that's what they're optimizing their efforts on.

So there's a lot of co-ordination. Usually you can't just rip out those silos and replace it with this brand new organization. But you do have, different organizations are built differently.

Sometimes growth teams sit within the product team. Sometimes they're autonomous and they'll sit separately. But ultimately what you're trying to do is co-ordinate the efforts of the full company, to maximize the number of people that have a great experience with the product.

Craig: I think that makes a lot of sense right out. You've got a 20 person marketing team. You've got a 10 person product team. You've got a 50% engineering team. You can't just say "hey guys, go all be cross functional now."

Sean: Right.

Craig: I think I've seen ways where you take a part of a product and said "hey, we're going to build a cross functional team across this." You own your own kind of marketing for acquisition of customers for this feature and there is some cross collaboration there.

In sense, well not necessarily a growth team. Kind of imitating some of that cross functional because it allows you to do things you couldn't previously. I think that makes a lot of sense. Because you mention there is a kind of cross functional growth team. Who's in that team? What's it look like? What's the make up of that?

Sean: The way I recommend companies build it is they start borrowing resources from a lot of people. So it might just be a single kind of product manager person who's running it. I like to call that person the growth master who's basically coordinating efforts across teams and trying to run experiments where they're borrowing resources for multiple teams.

But if they're finding that they can never get a designer to work with them. Then that might be the time to go and get a dedicated designer for that team. They can never get an engineer to work with them from the engineering team. Than add a dedicated engineer. I like to actually start not say, "let's define the perfect growth team. In six months, we'll be done building it and then we can start growing this way". I like to think how do we start growing this way today and what are the bottle necks to running the testing that is needed to ultimately discover better ways to grow the business. And then let's hire to fill those bottle necks.

Craig: Now you mentioned one thing in there where product managers could be the great, kind of spear headed this which of course because they're perfect for everything. But is this happen on a full time basis, part time? How does that kind of structure?

Sean: It's usually a full time role. You actually see the title in a lot of companies. Product Manager of Growth. It is a typical profile of the person who would be running it inside a company.

Craig: And the support they're getting is the rest of the team are they pulling of a designer kind of full time at that team.

Sean: That really depends so if a part time designer is getting the job done or apart of the designer's time is getting the job done. Then that's great. You don't necessarily need to put a new designer in there.

But most designers don't want to do prototype, fast work every week and so a lot of times a designer is somebody who's hard to get in growth teams.

So over time you start to have a designer, maybe some analysts even some marketing, and copywriter type people, products. Yeah front end, back end engineer. I mean there's a lot of kind of different things that can come together to form the team but it's again. I don't like to try to define the perfect team upfront. Because you just don't know what types of experiments you want to run.

Craig: Yeah, it all make sense.

Rimas: What are some of the risks that you find relative to building these teams out? Because we talked about pulling resources from other places. What are some of the perils and pitfalls that, if we were to speak a little bit more tactically someone that is trying to do this stuff is going to run into it?

Sean: So, you don't grow the business until you try stuff. Action is what leads to results and for a growth team, testing is the action. If it works then you do more of it. And so any one of the big risk factors that I see is that teams just spend a lot of time talking and brainstorming. And they go down these big kind of brainstorming rat holes and the more senior people they're bringing into that session. Eventually people start checking out and then they don't have the power to get things done because your VP of marketing wants no part of it.

You're VP of engineering wants no part of it and without their support, it's really hard to push the test through that you need done. I think that's where having a very structured process is important, where you're building a backlog of ideas. You're having a one hour very structured meeting each week of deciding which experiments you're going to run. You're focusing those experiments based on what the data is telling you or high leverage opportunities in the business.

So if you're having a retention problem is it because the product's really bad or is it because people aren't using the product right in the first place? That's usually the case with a lot of companies Is that one of the highest leverage areas? They should be focused on that first user experience and activation and so setting an objective around that. Running a bunch of experiments until you find that you're not improving as much. Then you move on to a different area so

I think that there is a very systematic process to being able to execute and grow and the learning curve for many companies is understanding the process involved.

Rimas: And does this apply to the single individual that's maybe at a start up. That growth PM that you're talking about, as well as the larger organization, that maybe is a 1000 person company. Maybe it has a small department doing this stuff.

Sean: So the super early stage company, you don't need as much process and structure. Just go out there and just start trying a lot of stuff. You see some signal, do some more of it. You can be pretty agile because this is a fairly agile growth process. Agility gets harder the bigger you get and so you need to be very purposeful around the agility.

Rimas: But the most important part though is the process. It is this cycle that you're going through.

Sean: Yeah but again I think that cycle,

It's okay to be a little sloppy in the super early stage accompanied with that process. Because initially you're fighting for survival. Most super early stage companies don't have a single channel that's working.

So LogMeIn would be a good example. I ran marketing there for five years and I initially was that guy. I was the sole marketing guy trying to grow the company. Eventually I added a couple other people in the early days but what I found is that I could not develop channels because we had such a bad activation problem. And so I got to the point where even just getting control of landing pages was really hard.

Craig: How far was it until you had that realization and how did you get to that? I'm kind of curious of how do you know if you have an activation problem?

Sean: So I was getting about a thousand people a day to sign up for the products. And then I was ultimately trying to optimize my spend on a dollar spent, do I get at least a dollar back? And what I found is I could scale to about 10,000 a month in spending. When I try to spend 11,000, I was wasting the money. And so trying and being super creative about that. I just could not find the channels to grow the business and then when I looked at "okay. I'm able to get a fairly reasonably cost person to sign up for the product. What happens after that?"

When I looked, 90 plus percent of the people were not actually using the product. And so if I don't get them to use the product that's like outside of my area of control. And the product team sort of has the product road map and they're thinking about all these other great features that we can do. So you got all of this marketing dollars just kind of being wasted in no man's land and so once we actually documented the path to using the product and where we were losing people it became clear that we were going to have a really hard time growing that business if these people are signing up and not using the product.

And so fortunately, we had a CEO who was super responsive when I presented the data to him and he said, "This needs to be a full company effort. This is the most important thing we're focusing on for the next few months." And we were able to get about a 1000% improvement in the number of people who actually used the product after signing up and what that did was free me up on the marketing side. I now could spend a million dollars a month on customer acquisition with a positive return on investment with no new channels.

So the exact same channels that previously scaled to ten thousand now scaled to a million. A lot of channels that just didn't work at all suddenly became viable now.

So I think that for me was the epiphany that marketing by itself does not work very well to grow a business unless everybody else becomes data driven. There's just this kind of inner dependency that happens between conversion, marketing, retention. That full customer journey needs to be right so that can actually compete in the channels.

Craig: Yeah, I think I've heard this from companies that have issues where there's this disconnect between marketing and sales and kind of product. Where marketing has their quota. Marketing qualified leads, Sales has theirs. "Hey we need more leads from marketing" and marketing just piping the funnel with whatever they can.

And it's like, "this isn't more qualified". It's just a lead because you can say you could have one. Sales is saying "I want better stuff in the funnel but I also want more" and then you've got product saying, "hey I care about lifetime value. I care about retention of customers. I get these people and they're here for three months, and they're gone because they're not the right ones."

And rethinking that. You mentioned you had a CEO there that you pitched this and he got it. That actually sounds like a fairly key thing to be re-thinking this because how do I come in and say "I have this problem", like maybe it's activation. May be it's retention. Maybe it's just acquisition. How do you reframe the organization around that do you just say "hey, here's this sone team. They can do whatever. How does that work?"

Sean: A lot of this is just scenario planning, where you just kind of say "where are we losing people in our funnel?" Just present your funnel. Where are the leaks in our funnel and what would happen if we plug those leaks? What will happen to our MRR? If we spent the exact same amount and now we converted twice as many people. What will happen to our MRR? Probably go up quite a bit, well if we SaaS in particular retention is such a huge part of SaaS.

If you have a 5% churn rate and you have a thousand customers. Every month you lose 50 customers and you need to add 50 customers not to shrink. So when you're the marketer in the SaaS business, you know that if there is a churn problem in that business, you're going to have a really hard time growing that business.

I think some of the stuff has just become more obvious as tracking has gotten better, and it's usually not a hard case to make. I think the hard part is re-orienting the business around the solution so that, the problem is not that hard to identify. The solution and implementing solutions is kind of tough.

Rimas: I have a crazy question. Have you found any teams that aren't data driven? That are trying to grow businesses and have been successful?

Sean: I mean there are definitely teams. Especially in some of the older businesses and marketers for a long time. Most online marketers are pretty data driven but there's the kind of creative marketers that say, "well we're building brand. We're not actually trying to grow customers right now or that's not our key metric. But awareness is gone way up or sentiment around the brand has gone way up." I think that they're driven by data but it's been a self serving data where they have maybe a little less accountability, the actual results in the business.

Craig: I've always wondered how a Procter and Gamble. How do they measure impact of the commercials? It's completely beyond my grasp being an online digital dead tools guy. They apparently have their metrics but it's beyond me on how you even measure.

Sean: Yeah I think about with beer commercials. What's that thing that cause the person to grab the Bud off the shelf in the 7/11 instead of the Coors Light?

Craig: Yep, completely.

Sean: I think that's one of the things that I find really interesting about super early stage companies is that you have a lot less variables. You have a lot more tracking and so for me, a lot of my career was just being passionate about actually running experiments and seeing what would happen and trying to figure out things that used to be in the John want to maker days.

I know half my budget is wasted. I just don't know which half. Today people know which half is wasted. They just don't know where to redeploy that budget to make it effective. So like they run out of places to spend money and not waste it.

Craig: We covered a good bit like it's identifying the experiments, running them, looking at the results and then where does it go from there? Do you just flip it on, keep doing more of that? How does that tie back into the rest of work? What's the long term role of the that kind of growth team as you develop. How do they tie back in or do they kind of carve out, and do they become their own functional team? How does that work in time.

Sean: The interesting thing that I've seen from product teams sometimes is that they talk about they want to have this great product experience but growth is not something that they're necessarily that interested in. And even founders sometimes and you start out trying to solve a problem because people are suffering and if you're not spreading that solution, there is no impact against that problem.

You've created a solution that's just collecting dust. And so I think that's kind of a mental shift across the organization is that people need to realize that as the goal. Once you've created something that solves a real problem for people, you want to solve it for as many people in the best way using this product.

And so I do think that there is a mental shift that happens across the whole company around growth, regardless of what is part of the growth team versus the product team. I actually heard at Uber a really good description that

The product team is about expanding potential and the growth team is about fulfilling potential.

It kind of make sense. I think they were saying it in terms of some of the new initiatives like UberEATS or that things that. The growth team is really about trying to push the envelope on that solution for better or worse with all the kind of discussions that maybe happening around some of those companies. But ultimately I think the idea of a North Star metrics is something that I think probably came out of Facebook, the first ones to really use it.

Where everyone within the company starting from the CEO said "we are optimizing for not just new people signing up for the platform, or not just accounts on the platform but monthly active users and eventually down to daily active users." And other companies that have been really effective with growth like Airbnb. It's nights booked but

They're essentially identifying the value creation event and trying to maximize those value creation events.

Rather than just new listings on the platform or new guests on the platform. But they're actually defining what creates value and value creates retention and retention is critical for driving long term sustainable growth.

Craig: Yeah, I think that you just kind of answered my next question right there before I could even get it out. It's like what's that metrics for me? For my business, it's kind of like "what is that value driving lever there that drives retention." That drive lifetime value so I think that, that's spot on if you know what that is. You're driving up overall the valuation of your company and what you're doing in the world.

Sean: And when I work with companies, that's the first thing that I do with every company that I've worked at. I'm still doing a little bit of consulting just to be able to tinker on companies and continue to develop my skill set. But the first thing that I do is I figure out who are the users who consider the product a must have. So I ask them one survey question which is "how would you feel if you could no longer use this product"? And I'm looking at the people who answer "very disappointed."

I ignore the somewhat disappointed. Very disappointed means it's a must have for those people and I'm trying to understand what is the benefit that they get from the product? What is the experience with the product that delivers on that benefit? And

How do I create a North Star metric that reflects an expansion of that experience and then how do I ultimately optimize the path to that experience and pour as many of the right people into that funnel as possible?

Rimas: Does any of this roll back into product road map and how does it inform that product road map?

Sean: It should. I haven't tried to tackle the product road map piece. I had a hard enough time with product people saying "what the heck are you doing?" You're a marketing guy.

Craig: Go back to marketing land. Leave the product alone.

Sean: If I said "let me give you some pointers on your road map", I think that would have probably got me thrown out the door pretty quickly. Interestingly product people more and more are starting to realize that there is some real value. And once you understand that the core value that the product delivers. Then doubling down and figuring out which features on the road map are an expansion of that value and which are kind of new beach heads.

Craig: So you mentioned like earlier on that product people are good kind of candidates for these early growth teams. This show is practical product. It sounds like there's a lot in here that product managers can take and just apply today even to existing product or just teams. If they're struggling to form this growth team or maybe they need to run it within their product team to get the buy in first.

What can they take away right away that's tactical that they can say, "hey I can do this tomorrow?" Just start to apply a lot of the principles. I'm hearing that a lot of the principles can be applied quickly.

Sean: Right so my big recommendations would be try to define what your North Star metric is and part of that is try to define what the core value of the product is, and then work backwards to what is the earliest part in the customer experience that delivers a taste of that.

Craig: Why earliest? Why not most important?

Sean: Earliest because it's time to value. There's so many things that people can be doing, so many products that they can be looking at that you're going to lose a lot of people in the period of consideration to value. And so you want to just understand this is what we call the "Aha moment or the point of activation." Facebook's kind of famous for their 10 friends in seven days.

They realize long term retention correlates to if they get somebody with 10 friends in seven days that person is likely to stick around long term on there. Once you know that as a top of funnel goal then you're doing everything you can to get people there quickly. Because it's easy to say I'm going to optimize for long term retention but it takes you months to figure out did it work or not and you gotta really like slow feedback loop on that.

But if you can work out to this Aha moment and data can tell you, and also intuition can tell you as well. I like to start with the intuition and then see if the data backs that up. And then ultimately that is kind of your first goal and then so we talked about what are the actions that somebody can do today.

Understand that must have experience and your North Star metrics. Understand that activation moment and then start to run tests.

You don't even have to change anything in your organization just hold yourself accountable for the number of tests that you're running. So Twitter, despite again, a company who's had some issues lately. Twitter actually flattened out in their growth in 2010, they had almost a completely flat quarter at the end of 2010. A new VP of products came in and said, "we run like one experiment or two experiments a month. That's not enough experiments to be able grow this."

And so he ended up cranking it up to bout 10 experiments per week. As soon as they did that, growth resumed for several years after that is

I think the biggest predictor of growth is actually "do you run enough experiments or not?"

And then everything after that becomes how do I figure out the best experiments to run? How do I get smarter about where I target the energy of running those experiments?

Craig: I'm curious to drill in a little bit to that too but first like for the smaller guys out there. Just trying to get started, not Twitter, 10 a week sounds exhausting. It's Twitter. I think it's a simple product. It's a 140 characters. What do you experiment with?

Sean: Slow down, slow down.

Craig: Don't get me wrong, I love Twitter. I'm on it constantly but I think about 10 a week there is a lot.

Sean: So I did the same thing on my team. Once I heard about Twitter, Satya Patel was the VP of product at the time and he was being interviewed in a product meet up. I was like, "huh, I'm going to try that in my company. So I said, we're not running very many experiments right now. I'm going to make sure that my team runs at least three experiments per week."

And so I said to my team, "guys we can't control the result. But what we can control is the input that leads the result." I don't care what experiments we run but any week where we don't run at least three experiments, I consider a failed week on the inputs that are going to lead to growth, and any week where we run at least three experiments. I'm going to consider a successful week. Soon as we started that, after three flat months. We grew like 60%, 70% on growhackers.com in the next 10 weeks.

Craig: How big was the team at this time?

Sean: Probably 12 people.

Craig: And could you give an example. How big are these experiments?

Sean: Some of them were super easy so that's what you're trying to kind of right size the experiments on what it takes to put them out. So like an Optimizely experiment that we ran. So we score each experiment on potential impact. How easy is the experiment and then how confident are we that it's going to work? And so on those three factors. The best one, you're super confident it's going to work. It's going to be really high impact if it works and it's super easy to run but rarely do those line up.

Rimas: Those run out really quickly.

Sean: Yeah.

Rimas: I think you've got one or two weeks of those and then now we actually have to run hard ones.

Sean: Right so I had one of my colleagues put in an experiment. We had an email collector on our site and his idea, he said,"I think there's an impact of 4. I don't think it's going to really move the needle that much. But we have it at the bottom of the page, let's try moving it to the top of the page." And with Optimizely, it's a really easy one to do. I don't think it will be that great but we have to do an experiment this week.

That gave us a 700% increase in number of emails collected per week. And it was really ugly in terms of the design of this collector. And so once we saw that big lift then I said, well let's actually put a designer on here now. And so our next experiment was redesigning it to actually make it look nicer.

Rimas: Did it go up then or down?

Sean: It did and it actually went up another 40% but what was interesting is that for all those people rushing out to move an email collector from the bottom page to the top. He actually moved it back to the bottom but he pinned it to the bottom. So when you're scrolling, it stayed kind of pinned to the browser but it was designed really nicely. It said which companies are subscribed to this weekly list and it had some good marketing copy on there, and so that gave us another boost.

I think that's kind of the idea with tests is that when something works. You double down, you do more test like that. When something doesn't work, you kind of mark it up. Okay we learned, we move on and so sometimes like pricing test for example. Give you an example of a really big test. I acquired a company from KISSmetrics, it was called KISSinsights at the time.

We changed the name to Qualaroo, we sold it a year ago but I acquired it under the assumption that these guys just don't know free-mium right and if I set free-mium right up on this business. It's going to skyrocket and so we just had to make the free version a lot my valuable, and then when we do that. I just had this whole model built out so I won't go into all the details but talk about a risky experiment. I spent like million plus dollars on this thing to then go and engineer it.

Craig: Where'd you go wrong?

Sean: And it didn't work at all. There was almost no increase in demand for the free version when we made the free version a lot better. All it did was sort of sucked demand off the paid version and so you could have said, "God that's the worst experiment ever." It was so expensive but what we learned was it's not really price sensitivity on this product. I thought there was price sensitivity. There's not people either need it or they don't need it.

Craig: So you can then just go back and go the other direction.

Sean: So kill the free version. Over time, we ended up increasing the price on the paid version 300% or 400% and a lot of our revenue growth ended up happening from these price increases.

Even a failed experiment leads to the learning that makes you smarter about different ways to grow the business.

Rimas: Is there ever a situation where you are running experiments and it actually takes you down this really dark rabbit hole where you shouldn't be. For example, where you're talking about all sorts of bad stuff.

Sean: Yeah I think that can happen with the product too. It's sort of like "oh jeez, if we just added gambling and porn to this product. It would be even more." I think people need to use judgment in everything that they're doing and I personally have not felt like I've crossed the line with anything that I'm doing. I wouldn't feel comfortable kind of crossing the line but my line is going to be different than other people.

It's subjective but the good news is that if you kind of listened to everything that I've been saying.

Most of the slimy things are the fleeting stuff. The stuff that doesn't actually work anyway. What I'm saying is that longterm growth is about understanding the value that you deliver and getting people to the experience of that value.

The most powerful experiments are ones that reduce friction to getting to that experience and so there's always going to be times where you have to make those judgements. But I rarely get in a point where I'm like "gosh, did we do that or not?" Just because my mind does not go there.

Craig: And I imagine kind of your growth teams even just product managers focused on growth that are thinking about this. One thing that Rimas really like to talk about is your product culture and your product principles and your guidelines and ethos. You understand that and you're shaping it around that and you're driving it again back to the value you are delivering as a product. How do you get people to that sooner and if it's oriented around that them it all aligns pretty well.

Sean: One discussion that people do have that's along those lines is Airbnb discussion of was it ethical when they did reposting onto Craigslist? So somebody posted there. People say the terms of service prohibited it. I don't know that. I guess I could go on the way back machine and see that it actually do that. Probably should, I'll do that after this.

But assuming that it didn't. I would say it's fine. To me that's where people are having a poor experience with finding short term rentals on Craigslist and Airbnb significantly improves it, and why not go to where the people are? But if they did have clear lines but they were violating them then I think it does cross the line. You don't want to violate those lines.

Craig: I think that we've covered a lot, dispelled a lot. A lot of good actually practical take aways. Okay, this actually sounds good. We included tactical tips that every PM should be digging into. I mean they learned everything that they need to in this podcast right so we're done. But if they want to go further. Where do I go? This is a new kind of way of thinking about cross functional teams, organizations, growth in general. Where do I dig it?

Sean: So hopefully we've at least kind of conveyed the principles here and sort of the why you might want to do this. It's not easy. I think one of the big risks that I do hear about. I haven't experienced too much myself but I've heard others is that when you're borrowing engineering resources as part of this or even you have dedicated engineering resources on the growth team.

Suddenly you have this big product feature push and everybody flocks to the product feature and basically you lose rhythm. When you lose rhythm, it's hard to get back in that testing tempo and kind of things fall apart.

The growth master role is really critical for keeping the rhythm going.

And we actually just launched a course at GrowthHackers called the growth master training course. Part of what we call growth university. So that's one way to learn more but I think in terms of just the overall process and kind of breaking everything. Borrowing best practice from a lot of companies. I have a co-author where we just wrote Hacking Growth and it's a very detailed how to book on all of this. I think it provides a pretty good reference to be able like where ever you might be hitting a wall. The book can point to how you can overcome that issue.

Craig: Cool, and I was lucky enough to get an early look at it. I think a lot of interesting, much more detailed practical advice in there. So I think it's definitely worth a look if you're curious to learn a bit more.

Rimas: I can't wait to check it out myself.

Sean: Awesome, thanks guys.

Craig: Yeah, thanks again for joining us.

Rimas: I totally enjoyed it. Thanks for coming on the show today.

Sean: Alright, thank you.