Ep. #3, Killing Features In Your Product

Craig Kerstiens: Thanks for joining us again this week. We're here with just myself and Rimas. No additional guests this week. This week we're going to dig into killing products or features.

It's easy to figure out, or pretty clear to figure out as part of product management, what you need to build and building the new thing. But part of that, I think, is actually taking away things and making products leaner, or just completely killing a product line altogether.

Rimas Silkaitis: Let me jump in here and say that killing products and/or features is something that I actually do not see a lot of discussion about out there within PM blogs or discussion forums or anything else like that. It's always about creating the new thing, and not taking away from what you have, to create a great experience in whatever it is that you're building.

Craig: You know, I think one big part of that, right off, is it's not as fun and exciting.

Rimas: Well, let's back up. I think it can be fun and exciting so long as you're curating the experience that you want for your customers. Sometimes that means taking things away that may not be used or really don't enhance what you're trying to do from a business strategy perspective.

Craig: Now, you're already kind of stealing my first question there, which is, why do you even care? It's really easy to keep adding onto a product. If you add more features, shouldn't it add value for more people?

Rimas: Oh, simple logic would dictate that, yes. Just keep adding stuff, and you'll be fine. But I think great examples of that are things like Microsoft Excel. I think that thing has so many different features in there, that I don't even know what it can do anymore.

Craig: Where do you even define that balance? Early on in this stage, I think for a lot of companies starting out, they don't have to worry about this. When do you start to hear about it?

Rimas: I'll bring up a good example from my own work right now with Heroku Postgres. What we do is that for every subsequent version, there's always a new release on the platform. So there's Postgres 9.1, Postgres 9.2, 9.3, 9.4 and 9.5, and for us there's an operational burden associated with maintaining all of those versions.

At some point, that means that we have to pull one of those out, deprecate it, and that means customers can't use that anymore. Granted, there's always new and exciting stuff in Postgres, but even the Postgres community will have to deprecate versions or stop supporting them. That means no security updates, etc. That means that I have to then subsequently go out and remove, let's say, 9.1 from my supported versions.

Craig: That makes a lot of sense. I get it, Postgres, end of life. Ruby, end of life. Some language, some service that you don't have control of is end of life. The cost to support it would be pretty high. What if you've already built it? It's a SaaS product, there is no overhead to maintaining it. Why, is there a reason to ever kill a thing like that?

Rimas: There's always an overhead to maintain something. You can never say that there's never anything that have to do. Maybe from a technical standpoint, if you've launched this product and it's running just on it's own, but there's always going to be something like customer support, business operations, something else that requires you to spend time on this.

I don't think you can ever say that something is just kind of running on its own without you ever having to spend time on it.

In those kinds of situations, it really depends on what you're trying to accomplish from a business perspective, and why or why not keep this thing around. Because if it's not doing anything for you or contributing to some goal that you have, whether that may be your brand, your business, revenue, costs, whatever, then maybe you should just get rid of it.

Craig: I think you kind of hit on something in there too, where you're headed as a business. That's an important point about the future. It's easy to build a product for the customers you have today, and as people, we generally don't like change. Customers especially don't like change.

When they're paying for something, they don't want it to change, even if they're not paying for something, they generally don't want it to change. So as you advance the product and kind of move to a bigger market, move to where you're headed, there's an opportunity there that you need to keep customers coming in that direction, which sometimes means killing something that is incredibly popular. That's usually kind of one of the hardest decisions, I think, to make along the line.

Rimas: So, what factors for you will dictate when to kill something? I mean, I've mentioned business and I've mentioned strategy a little bit. I think we're missing a little of the customer stuff. But how about for you?

Craig: I think you hit on it a little bit. What is the cost to maintain it, what is the burden? That's a big one. Usually I view it as kind of a percentage of your time. If a certain amount of percentage of your engineering staff is spent just maintaining a thing, and it's not actively growing, it's kind of flat, to me, that's an immediate red flag that I'm going to dig into.

Rimas: But if it's not growing, are you not spending enough time on marketing or sales or something else like that?

Craig: Part of it is in the broader context of the rest of the business. If the other areas of the business are growing, if other features are being used in our driver for growth, then it makes sense to double down more on those than try to just kind of even it out across the board.

Rimas: It sounds like there's this proportionality between how much money this thing is bringing in versus how much you're spending to actually maintain it.

Craig: A big part of that is really going to be engaging with your customers. You need to know what are they using, what are they not.

You can do some of this with data. Some of this you should be measuring which features they're using, which ones they aren't. You should also be interviewing them, talking to them around what influenced their buying decision. It may not be an active feature, but it may have been a big factor in the buying decision.

Rimas: I think that's a good point, and I think in a previous episode, our last one, where we talked about enterprise, selling, etc., how you interact with those customers is going to be a lot different than you would with consumer based products. So don't forget to talk to those customers.

Craig: I think you look at the data there. You may have one user, but that may account for the one reason they use the product, and that may be 80% of your revenue. So you do need to look at the big picture of it all. There's not a perfect science to, "This only has one user, we need to kill it."

Rimas: That actually touches on the next thing. What happens if there's no negative feedback? Like in the case in point, you've got that one customer that's 80 percent of your revenue.

Craig: I think, from experience, when you kill something, you worry very heavily about the backlash. You sit there and you monitor Hacker News and read every negative comment, and you kind of feel that pain. You read all the negative ones on Twitter, which you should be doing, you should be looking at that negative feedback.

If there is no negative feedback, I've made the mistake in the past that I was saying, "Well, you know, it was perfect. It was no problem." Over time, I think you've got to be careful about how much you wear on customers, how much you cause them to change their workflow, how much you change the product for them when they weren't expecting it.

It's not one feature usually that causes you to lose a customer, it's five features every month, that they love over time, that gradually erodes that trust that you have between them.

Rimas: I like that word "trust." Because when you're working with a customer, they depend on your software or the product that you've built to do something. And if they cannot accomplish their tasks, whatever they may be, or the problem that you're trying to solve for them, then yes, you're going to erode that trust. And they are less likely to recommend you as a solution going forward.

Craig: I think it gets pretty quickly into one of the next topics, and we can come back in a minute, but that trust I think is important. And it really comes down to how you do it. There are a lot of pieces in that around how do you communicate the timeline. What makes sense there? When you're killing off a version of Postgres, when you're killing off a feature in a SaaS app, how do you actually go through the process?

Rimas: First, I know for me, I always think about how long it might take myself to do something like that. Granted, that is not indicative of all the customers that I have, but what will allow me to think about this is what's it going to take to go from one version to the next. What are the roadblocks, barriers, gotchas from going from, let's say, Postgres 9.1 to 9.5. That's a huge jump, there are a lot of gotchas in that whole thing.

What would a customer have to do to be able to get there? And one of the other things that I think about, is how can I make it easier for them to get from one place to another? Sometimes that requires building even more stuff to get there, which is okay because if I can automate it, then that means I can actually compress the timeline more.

And that's actually what I'm looking for anytime I'm doing any kind of deprecation or killing features. Because the longer I keep that timeline out there, that is just more on point that I have to be and the team has to be to be able to manage all the expectations around that deprecation once it's been announced.

Craig: I think there are a couple of pieces in there. There's "how long does it actually take them to do the work?" It takes me a week to do the work. So when am I going to start doing that work? Probably less than a week before you kill the thing, because I'm going to wait until the last minute, and then I'm actually going to be kind of behind the eight ball.

At that point, do you build in that cushion in timelines? Like, you say we're going to kill this feature in a month. It's going to take me two weeks to migrate. What happens when half the people haven't migrated in two weeks out?

Rimas: When half the people haven't migrated?

Craig: Yep.

Rimas: Well, it's like a bell curve. You're always going to have the early people that are going to say, "Holy smokes, I've got to go and get off this version immediately."

Craig: Can I have your customers? Because I'm not too familiar with those.

Rimas: That's why I said it's like a bell curve. These are like the early adapters. The small minority of individuals and companies that recognize that when something is being deprecated or killed, they'll have to move off of this and take proactive steps to do so.

Then you've got this large majority somewhere in the middle that may or may not do stuff towards the end of this timeline. And then you've got the laggers who just won't even touch anything or do anything relative to a feature killing, which means that you actually have to make the hard decisions about what do you do with their data.

Do you just get rid of it? Do you put it somewhere else? How do you communicate to them that, "Hey, this is your last chance. This thing's going away"?

Craig: I think we're talking a lot about the timeline. The sunset process already presupposes there is a timeline. Is there a case for just "Shut the door immediately"? I mean, especially in the SaaS world, data is hard to do. But in a SaaS world, can't I just remove the feature?

Rimas: It depends on what kind of issue this is causing for you as a business, personally, as a product, etc. Let's say you release some kind of feature that's causing collateral damage, meaning data loss or customers are negatively impacted in terms of their business. Yes, shut it off immediately. Sometimes that can be the best approach, but as with anything on this show, it's always "it depends."

Craig: That's a perfect slogan for "Product." I think it makes a lot of sense. If it's actually doing active damage, shutting it down immediately, taking proactive steps, really makes a lot of sense there. I think the other one we commonly see is a company didn't succeed, it's going under, that one there's not much of a choice. It's kind of the Thor situation all around.

Rimas: I would even caution there and still say I would do what I can to set things up and make it right for those people that put that trust in you. Because yes, killing features is not sexy, it's not something that a lot of PMs do.

There's still glory in actually doing it the right way so that customers can get their data out or move on to the next thing or whatever it may be.

Craig: And I think it's a big one as you're shutting down a service; it's not commenting that you need, really want to practice how to do, but making sure there's a smooth landing for someone else. Whether it's a competing service, whether it's a similar kind of product, making that transition as smooth as possible is really key for your customers, and they'll value you over the long term for it.

Rimas: Let's point out there, they'll value you. The company may have gone away in a lot of these cases, but you have your own reputation to deal with outside of this company that you're shutting down, or feature or whatever else. And by doing a great job of that, that's still building up a good reputation for yourself.

Craig: Yeah, which I think does have a positive impact for long term. Coming back a little bit to that timeline that we kind of jumped right into. What does a timeline look like for you? Is it just "We're going to pick the date and we let people know about it"? What's the process there?

Rimas: Well, I usually just pick a date. I mean, sometimes you just want to say, "Okay, let's just do this at this time." Typically, there's other competing projects or things that are going on within the company that dictate whether or not it happens within a planning cycle or something like that.

If you're a smaller company, again, just like we had talked about before, consider what it would take for the customers to get off this stuff, and then work back from what it would take to then dictate whatever that timeline might be.

Craig: I think the urgency with which you want to do it is one factor. What are the costs involved? Don't pick a weekend. Just think through some of these things. Usually middle of the week is good, holidays are bad.

Rimas: I usually like Tuesdays. Everyone has gotten Monday out of the way, and usually on Tuesday everyone is kind of looking for the next thing, and you've got the rest of the week to deal with comments.

Craig: All of the stuff that they didn't expect to handle, yeah. I think that's a great one. I mean, holidays are a common one that people overlap, and especially when it's a far out timeframe. Summers are hard, too.

Rimas: Especially, yes. Those are terrible. I actually prefer the fall over anything else, because everyone's in the mode, working, doing their thing, and they're not on vacations.

Craig: Yep, I think that's important one. Think about when your customers are around, when they're available, and the specific date you pick.

Rimas: The end of the year is also pretty challenging, I will say. With holidays, New Year's, etc., no one's around. Good luck getting in touch with anybody.

Craig: Absolutely, they're not even going to notice it. They're going to come back to a pile of email, then half of staff is around, and they're just keeping the lights on. It's really probably the absolute worst time you could do it.

Rimas: So, talk to me about communication. You've determined this timeline. We've talked about a few of the factors: urgency, the time you want to take, just picking a date out of thin air, if possible. How do you communicate to your customers that this is happening? And how do you manage that?

Craig: For me, it's really like a one-to-one kind of communication with them.

Rimas: So you're calling every customer up?

Craig: Calling or email. If it's a SaaS product, email works great. You're blog is not an outlet for communicating a shutdown.

Your blog's great for marketing when you have a product announcement, and it's fine to have it there and have it be documented. But don't assume that it's on the blog that people read it.

Rimas: My SaaS app has 100K, 500K users. You're telling me that I have to go and email each one of those individually?

Craig: You know, I think it depends on the shutdown we're talking about. If it's a major product, you should actually know if the 500K people are using it. Coming back to data, you should reach out to the people that are using it, or have used it in the past, directly. If you can do this in product, that's fine as long as you know they're aware of it.

But putting it up on a blog post, or putting it within your app but not actually tracking if they ever saw it, then that's not communicating with them. So you need to make sure you have a message to them that will get visibility.

Rimas: It sounds like you're recommending to me that I might use some marketing tools or something else like that. Maybe something to track that my emails are actually going out and getting read.

Craig: I think that this comes back to data. You should, if you have the data that you know you're killing this product or this feature, you should have the data to be able to send them a message and say, "Hey, they actually know this is coming, and they're not surprised by it."

Some people will probably be surprised no matter what you do, but you should be able to gauge the reaction when you do finally do this actual shutdown and turn things off. If this is 50% of people, then you didn't do the right job in communication.

Rimas: Phone calls, tell me about that. Have you had to call anybody about this stuff?

Craig: I haven't. I always pawn it off to sales.

Rimas: Well, that's actually another good point of communication here, that we brought up earlier in the show, that it depends on where your product is. We keep talking about SaaS apps, but there's consumer, there's enterprise, and when you're in enterprise, you have these sales teams and account managers.

You definitely need to get in front of them as soon as possible when you know you're going to deprecate something or kill a feature, because those individuals are the face of your organization, and if they're not prepped for what's coming, they're going to get a barrage of emails, phone calls from their customers going "What the heck's going on? I found out about this via some channel," maybe it was the blog if you did it poorly, saying "Hey, I find out this thing's going away. We rely on this. What the heck's going on?"

Craig: You really need to have your ducks in a row with them. You need to say, "Here are the questions you should expect to get, here is how we're replacing this or adding new value." You need to give them something that they can go to a customer with and to have it be a positive engagement versus a "Hey, this thing that you like? We're taking it away. I'm the bearer of bad news."

You really want to make sure you have that positive relationship with sales and they have that with the customer, so you need to equip them as best as possible well ahead of time.

Rimas: Do you tell the accountant sales teams about this change prior to actually publicly announcing it?

Craig: Oh, absolutely. Now, you don't usually give them the choice to weigh into it, necessarily. As the product owner, you own that product. Sometimes you can capture input from them, but if you are making a decision, don't let that be swayed in that sense.

Rimas: Okay, so just to set up the scenario then, I go to my sales. I've decided to do this, I'm going to do it, set up some dates, before I make the public announcement. I go to my account and sales teams, tell them about this, and then they automatically go and tell their customers, and then we do the public announcement.

Craig: I think it depends. Usually there are some premier accounts that they may want to handhold. This is probably not every account of theirs. Usually, that's in my experience, is they're engaging with some of the really premier accounts, and this is definitely on a bigger feature shutdown.

If I'm moving a button from one side of the page to the other, we're not going through this process. But on the bigger ones, on those premier accounts, and that's a conversation where if I'm working with sales, they are probably only looking at their accounts. What I'm trying to do is get a picture of all of the accounts, "So, are we reaching out to 20 people or 10?" They may all 10 fall under one sales guy. He may have all the big accounts.

Rimas: I could imagine that some of the customers as well might need some face time as a result of this, depending on what you're sunsetting. If that's required, just keep in mind all the travel time and everything else that has to go into that to talk to those customers.

I like where you're going with the talking to sales and account first, and then those customers, and then the public announcement. I think that's a great approach.

Craig: I think it's one key thing that, throughout this process, you need to try to feel their pain as much as possible. Yes, you're doing the right thing in the long term, but people don't react positively to change. People don't react positively to thinking they're one number.

That actually reminds me of a great quote from a former colleague, that he told customer about a change and said, "Hey, but this is positive for 90% of people." The customer wasn't in that 90 percent of people. They didn't react too happily to being viewed as just a cohort where this was a negative experience for them.

Rimas: Oh, that's terrible. That's too bad.

Craig: You really need to think about the sales team being able to have that empathy and relate to the customer, prioritize their discomfort, so that you're conscious of it and working through it in the right way.

Rimas: So how do you know that this was a success?

Craig: That's a good question. One is, are you able to execute it? Do you roll back ever? I really, really hope that's never the case. On the date front, there's often a couple dates there. There's the time you say you're going to shut down, sometimes there's a little extra wiggle room there. If you're actually de-provisioning something or completely ripping out the code, that hard date still has to come.

Rimas: I'm glad you brought that up, because I know when I've killed stuff, I actually like to have "This is the official, and we're cutting this off." But a lot of times, if customers actually write in or talk to me personally. I'll actually make that a soft cut-off date and give them another month or so if they've engaged to say, "Okay, I'm going to turn this off."

Craig: The thing I think you do have to be a little careful about there is, are you mis-setting expectations for the next time around, the next time around, the next time around?

Rimas: Like you'd said before, it's a hard cut-off date at some point. And for me, I know that if people come in to me and say "I've submitted a support ticket, I need a little extra time because I want to do this myself." I'll say, "That's great. I'll give you one more month, but at that time, I really have to shut it off, and it will be automated at that point. So I'm giving you that extra time in earnest, in trust, but this goes both ways, customer."

Craig: Coming back to "what is success." You've made that hard cut-off, you've ripped out code, deleted it. One big tradition at Heroku was having burn parties, actually burn code that we had deleted. I've seen binders printed out of code that we were burning before, a nice kind of symbolism of killing the old. So I think that's one big piece, are you actually able to do that?

The other is looking still at a holistic view of engagement. Intercom is really good at talking about this, where just because you launch a feature, it may get a lot of activity and pickup. But if that activity and pickup just came from another area of your product, and you didn't increase overall time in your product, if that's one of your metrics, then it wasn't really a success.

You could have just done nothing, and they would have kept using the other thing and been perfectly happy. So, with killing a feature, are you increasing overall time in your app? Are you losing overall time in your app? Did you have people leave?

I think it's important that, if you still have a vision, you kind of know where you're headed there. But is it conscious to be tracking what are you doing to attrition, to user engagement, as you remove that feature?

Rimas: But you should know all this stuff prior to going into this de-provisioning, killing, whatever it may be.

Craig: In a perfect world, yes. I think product isn't always in a nice textbook, so sometimes you're making some calls around, "We have this vision, we want to get here from a roadmap perspective. This is where we're headed towards." And there may be some bumps along the road there, and I think it's important to be measuring those.

Rimas: Okay, so I might have some goals prior to this de-provisioning, but some things might come up, and I'll have to make some decisions. Fair enough.

Craig: How about yourself? What do you consider a success when all is said and done?

Rimas: I think success, for me, means, have I done right by the customer? Have I given them an outlet to do whatever it is that they need to do? Because by and large, my customers use my products to accomplish something. They have a problem that needs to be solved, and I'm helping them do that.

If they are unable to do that, then I've failed my customers, because I want them to make sure that they can accomplish whatever it is that they need to do in their businesses and whatnot. They do not spend their time worrying about what my product does so that they can stay up to date.

Craig: One other important thing is the overall morale of the customer. You talk about them using the products, using the feature; are they able to do what they're trying to? Over the long term, you have to be careful about killing products, because it does erode trust over time.

It's kind of that death by 1,000 paper cuts. It's not the one feature you killed, it's not the second one. Maybe it's the fifth, and then I happen to get an email from your competitor that week.

So you have to be careful there of how is that trust eroding and measure that over time, whether that's net promoter score or some other mechanism.

Rimas: One of the things I like to do when it comes to killing features is actually have some kind of cadence for doing so. We talked about things that are end of life-ing, in those scenarios there's usually some kind of cadence that you can bring to it so that customers have an expectation of, "In the next year, this is going to happen again in this version."

With things that, like SaaS products where it's less so, where you have a feature that may not be working out, again I think you need to be cognizant of when you kill these things. Because, like you said, doubt, death by 1,000 paper cuts, you've got to give these things some kind of cadence, otherwise customers have no expectations of when things are coming or not.

Craig: Yep, and I think especially on the SaaS side, as you remove something, there's often something new coming along.

Rimas: I hope. Maybe.

Craig: Right, so as much as you can kind of bundle some of that together, make it a smooth experience, smooth transition, you talked a little bit about this, basically I can make it seamless for you so that you don't actually have to do something and something changed underneath you, but in a positive way, that's the perfect world. I think that's when you really have a successful killing of a product.

Rimas: For me, that's a product principle. That's something that I abide by, that is something that not everyone else has to abide by, but I'm recommending that if you're listening to this show, you do the same.

Craig: I think that's a pretty good principle to end on for this week. Any other final thoughts? I feel like as a general principle, I think we both strongly believe that this makes sense and gives you a better product experience.

It allows you to really control and cater and not become everything kind of in the toolbox. It's one small, sharp tool which, especially for developer tools, makes a lot of sense, that Unix philosophy. As much as you can, lean your product and kind of clean it down. It makes a lot of sense.

Rimas: I think the thing I would say to end on here is that killing features and products can actually do right for not only your company, but for your own career. So as you consider doing that, make sure it's a great experience, because you're only harming yourself in the long run if it's a bad one.

Listen to what your customers are saying. You should be talking to them all the time anyway. Make sure it's a good one, because it'll pay dividends in the long term.