Alistair Croll and Benjamin Yoskovitz wrote the upcoming book Lean Analytics: Use Data to Build a Better Startup Faster. In the following interview, they discuss the inspiration behind their book, the unique aspects of using analytics in a startup environment, and more.
What inspired both of you to write your book?
Alistair Croll
A big part of the inspiration came from our work with Year One Labs, an early stage accelerator that we co-founded with two other partners in 2010. We implemented a Lean Startup program that we put the startups through and provided them with up to 12 months of hands-on mentorship. We saw with these companies as well as others that we’ve worked on ourselves, advised and invested in, that they struggled with what to measure, how to measure it, and why to measure certain things.
The core principle of Lean Startup is build, measure, and learn. While most entrepreneurs understand the “build” part since they’re often technical founders that are excellent at building stuff, they had a hard time with the measure and learn parts of the cycle. Lean Analytics is a way of codifying that further, without being overly prescriptive. We hope it provides a practical and deeper guide to implementing Lean Startup principles successfully and using analytics to genuinely affect your business.
What are some of the unique aspects to using analytics in a startup environment?
Benjamin Yoskovitz
One of the biggest challenges with using analytics in a startup environment is the vast amount of unknowns that a startup faces. Startups don’t really know what they are at the beginning. In fact, they shouldn’t even be building a product to solve a problem. In many ways they’re building products to learn what to build. Learning in an environment of risk and uncertainty is hard. So tracking things is also hard. Startups are also heavily influenced by what they see around them. They see companies that seem to be growing really quickly, the latest hottest trend, competition and so on. Those influences can negatively affect a startup’s focus and the rigorous approach needed to find true insight and grow a real business. Lean Analytics is meant to poke a hole in an entrepreneur’s reality distortion field, and encourage…or force! … a level of focus and attention that can cut out the noise and help founders move as quickly as possible without doing so blindly.
What defines a good metric?
Good metrics have a few qualities. For starters, a good metric should be a ratio or rate. It makes the number easier to compare. You want to avoid absolute numbers that always go up and to the right. Those are typically vanity metrics.
A good metric has to be incredibly easy to understand. You should be able to tell anyone the number and they can instantly understand what you’re doing and why.
A good metric, ultimately, has to change the way you behave. Or at least provide the opportunity for you to change. If you’re tracking a number and can’t figure out how changes in that number whether it be up, down, or sideways, would impact how you behave and what you do, then it’s a bad number. It probably isn’t worth tracking and certainly not worth focusing on. Good metrics are designed to improve decision making.
What are the stages of lean analytics?
We’ve defined five stages of Lean Analytics: Empathy, Stickiness, Virality, Revenue and Scale. We believe all startups go through these stages in this order, although we’ve certainly seen exceptions. And we’ve defined these stages as a way of focusing on a startup’s lifecycle and how the metrics change as a startup moves from one stage to the next. We’ve also created gates through which a startup goes to help it decide whether it’s ready to move to the next stage.
Empathy is all about “getting out of the building” and identifying problems worth solving. It’s about key insights that you’ll learn from interviewing customers, which guides you to a solution. The metrics you track here are largely qualitative, but you may also start to look at levels of interest you can drive to a website or landing page and early conversion. Basically, you have to answer the question: Does anyone really care about what I’m doing?
Stickiness is about proving that people use your product which early on is a Minimum Viable Product or MVP and that people remain engaged. You’re going to track the percent of active users, frequency of use, and try to qualitatively understand if you’re providing the value you promised to customers.
Virality is about figuring out and growing your acquisition channels. Now that you have a product that’s working reasonably well with early adopters, how do you grow the list of users and see if they too become active and engaged? The metric to track here is viral coefficient which in a perfect world is above 1, meaning that every active user invites one other user that becomes active, in which case you can grow quite quickly, but it’s not the only metric that matters. You want to track actions within your application or product that are designed to encourage virality. This might be invites or shares. You have to look at the difference between inherent and artificial virality as well. Ultimately you get through this stage when you’ve proven that you can acquire users reasonably well, and you see scalable opportunities to do so going forward.
Revenue is about providing the fundamentals of the business model. Prior to getting to this stage you may have been charging money, but you weren’t focused on fine tuning that aspect of the business. And you were properly spending money to acquire customers but not really focusing on whether the economics made sense. Now you have to prove the economics. So you look at things like the Customer Lifetime Value and compare that to the Customer Acquisition Cost. You might look at the Customer Acquisition Payback which is how long does it take a customer to payback the acquisition cost you made to bring them in. You’re likely going to look at conversion from free to paid, especially if you are building a freemium business. You’re also going to look at churn or how many people abandon your product or service. To get through this stage you need to have a reasonably well-oiled financial machine that makes sense.
Scale is about growing the business as big as possible. You know you can acquire customers, you know a good enough percentage will stick around and pay, and you know the economics make sense. So now you have to grow. Depending on your business you’ll be looking at different channels such as partners or growing a bigger sales team, APIs for developing an ecosystem, business development opportunities and so on. You may expand into new markets, develop secondary, or ancillary products as well.
The book is filled with case studies. How did both of you decide which case studies to include in the book and why?
It wasn’t a complicated process. Many of the case studies came from people we knew and who were leveraging Lean Startup and analytics in a meaningful way. Some of them came from our own experience. We felt it was important to share those as well. As we developed the framework for the book, such as tackling different business models, the Lean Analytics stages, etc., we looked for great examples that could speak to each of the key points we were making. We talk a great deal in the book about The One Metric That Matters. This basically means: focus on one metric only, at any given time. This came from our experience but also from talking to a lot of other people. Then we picked a couple of great stories or case studies that reflected the importance of the concept.
It was important for us to have real world examples of all types of companies whether they were big, small, successful, less so, early stage, late stage, etc., so there would be variety, but also because we know these examples resonate a great deal with people. We know that people are looking for “proof” that Lean works and that a focus on analytics matters; hopefully we’ve been able to provide that in the book.
This interview was edited and condensed.