Government Really Needs Lean!

And so here we go again. Another government minister accused of a "humiliating climb down". Michael Gove is being lampooned in the press today for altering the implementation of the proposed changes to GCSE/High School exams in the UK. The opposition parties are loving it. Its a chance to throw rocks at the throne. The press love it because it makes for dramatic headlines and sells papers.

However, from a Lean Startup perspective this is patently ridiculous. The notion that government ministers cannot change their minds after considering the evidence is strange. This is exactly what we want them to do. It is the sign of a healthy democracy.

The idea that any person in this world can come up with a perfect idea that can not be improved  changed or even dropped is ludicrous. This is the same practice that makes bankers and investors expect startup founders to write business plans with precise five-year projections. It's crazy!

All ideas, including those from government ministers should be considered tentative. Then, each idea should be systematically tested against reality to see if it is implementable. If it is not implementable it should be dropped or changed. Even during implementation, government departments should be allowed to iterate ideas in response to the market. And the press should be lining up to congratulate any minister who does this. Otherwise you encourage people to always fight their corner in defence of terrible ideas.

We need more startup practices and agility in government. Or may be I am hoping for too much!


  

That Would Never Work in My Startup…

One of the most common reactions I get from people when I am running workshops or mentoring teams on the Lean Startup methods is:

“I love the concepts, they are really wonderful…

But that would never work in my startup”

A range of reasons are normally given for this reaction. These include:

•   My idea will get stolen.

•  I don't have a patent yet.

• My idea is too radical!

• People won’t really understand my whole offer until I build the whole thing!

• A minimum viable product will ruin my brand.

• We are just like Facebook, but for older people….  

There are more reasons, and often very few of them are any good. At a recent talk I gave, I was challenged to come up with a minimum viable product that would test the viability of a company selling gold-plated donuts. The argument was that you need VC money to buy the gold, before you can make the product and then test it. He almost got me too… I am standing there racking my brain, and then my good friend Liam Gooding (CEO at Virally) stepped in with a great answer:

“Well, you just get boxes and label them ‘Gold Plated Donuts’. 

Put bricks inside and go to the main street in town and try to sell the boxes for $50 each. 

And every time someone pays you, you say thank you very much, 

I am just running an experiment and give them their money back”

Awesome!

It is still important to emphasise that although Liam’s answer is a fantastic example of a minimum viable product  experiments should never be designed without a proper falsifiable hypothesis being stated, along with the minimum success criteria being set.

My argument is not that Lean Startup applies to every business in every context. But regardless of who you are and what business you are in; if you make something nobody wants your business will fail. As such, it just seems like a good idea to find out as early as possible if you are making something people want. 

I am curious to hear about other reasons that people have heard as arguments for why lean does not apply to a certain person’s startup or business. Please share below… 

Oh Those Lonely Assumptions – They Will So Return to Haunt You

As a metaphor for innovation teams, the marshmallow challenge is fantastic. Teams compete to build, within 18 minutes, the tallest free standing structure with spaghetti sticks and a single marshmallow placed on top. The best way for teams to build a good structure is to quickly begin experimenting with various prototypes, each time making sure the marshmallow is on top. However, very few teams do this. Instead, most teams spend a lot of time planning, and then build their structure without ever touching the marshmallow. Only when time is running out do the teams remember the marshmallow. They place it on top of their spaghetti stick structure and drumroll…., the structure collapses!!

Last month, we had a group of 20 Italian students visit the University of Kent for a week of learning about Lean Startup methods and business model design; (i.e. Valideation Camp). The students were undergraduates studying various degrees at Italian Universities (e.g. economics and engineering). The programme was funded by Fondazione CRT (http://www.fondazionecrt.it/), a non-profit organization setup by the Italian banking sector. These were the smartest students selected through a rigorous process. As such, expectations were high.

Their first task on the first day was the Marshmallow Challenge. The four teams were all set and ready to go. We noticed a pattern quickly emerge as they were doing the task. The first thing we noticed was that they spent a lot of time planning. Some even drew elaborate designs of what their structure would look like. In fact, one team was planning for over 10 minutes (for an 18 minute task)!

This behaviour is the classic throwback to business school teaching. You need an elaborate and detailed business plan before you can start working on your business. This cultural focus on elaborate planning can be problematic when you are uncertain about the factors that will lead to success on a task. To make any plans in an uncertain context means you have to make a lot of assumptions. If you don’t give yourself enough time and structure to test your assumptions, and any of them turn out to be wrong, whatever innovation project your team is working on will fail.

When the teams eventually started building their structures, none of them built prototypes using the marshmallow. They just built their spaghetti stick structures. The consistency of this behaviour was so surprising to me that I walked around from table to table and took pictures of these lonely ignored marshmallows sitting on the table.

When we finally called time, all the team scrambled to place the marshmallow on top. In the end, only one team succeeded in having free standing structure, with the marshmallow on top. And the way that team had built their structure was ethically questionable! They just bunch the sticks and stuck them to the end of the table!

As such, overall the whole task was a failure! The teams made the fundamental mistake of making a plan and just executing to the end without testing it first. In their planning, the teams made the flawed assumption that because a marshmallow is soft and fluffy, it must be light enough to be supported by their structures.

Oh those lonely assumptions, how they return to haunt you!

This is a question all entrepreneurs must face. As you are executing your startup idea, what assumptions are you making about customer problems, customer jobs to be done, channels to reach your customers, revenue models and marketing strategies? Of those assumptions, which ones are so risky that if they turn out to be wrong, the ‘structure’ of your startup will collapse? It is much better to stop executing an idea based on assumptions and get out of the building and start testing your assumptions now. You don’t want those assumptions to come back and haunt you much later in the innovation process. Often it’s then too late to make the necessary adjustments and changes.      

A Startup is the Founders’ Thesis

Since publishing my last essay critiquing the representation of startups as an experiment, I have been racking my brain thinking of what the equivalent of startups within science might be. This representation would have to be large enough in scope and impact to represent the unique contribution that a truly successful startup makes to the lives of people and society in general. Then it suddenly struck me:

A startup is actually a lot like a PhD thesis or dissertation.

 This may not be as catchy as calling startups experiments, but the similarities are so strong that it’s strangely uncanny. It also provides a way to fully align an entrepreneurial management science with the scientific method. Below I outline four core similarities to illustrate my point:

1.  A Path for Searching

Here an interesting etymology. The term “thesis” comes from the Greek word meaning “study”, and the term “dissertation” comes from the Latin word meaning “path” (Wikipedia, 2012). The word “study” is consistent with Steve Blank’s proposition that startups should be designed to “search” for a sustainable business model through customer development and customer discovery. Steve’s proposition is simply that the job of startup teams is to “study” the market and discover two main things: 1) What customers want and; 2) A profitable business model for delivering value to customers. In science it is the same; scientific study is really just a “search” to understand how the world really works.

Within science there are clear guidelines of what constitutes a good “path” for researchers to follow when conducting their thesis (i.e. the scientific method). Within startups Eric Ries has developed a similarly powerful concept. The build-measure-learn loop provides a clear “path” for startups to take as they conduct their “search” for a sustainable business model. The power of the build-measure-learn process is that it acknowledges that you will go through various iterations of your initial ideas before you find the “truth”. In science, it is the same. There is an acknowledgement that during your dissertation, you will go through various permutations and conduct several studies until you reach your destination.

2. Research Project

The power of the thesis metaphor lies in the idea that a startup can actually be viewed as a research project. Rather than a business plan, the startup idea is a research proposal. The problem with the business plan is that it orients startups towards execution because it is, after all, a ‘plan’. And we love it when a good ‘plan’ comes together. The term ‘research proposal’ is much more tentative. It is just a proposal; meaning ‘we could be wrong’. This orients startups toward learning whether they are on the right path. This is exactly parallel to the orientation that scientists take at the beginning of any new research project.

3. Research Team

Within the thesis metaphor, startup teams can start to view themselves as research teams, rather than teams set up to deliver a product. A few months ago I had an argument with the lead engineer of a startup who viewed his role in the startup as being; “to build the best technology possible”. So he spends all his time reading software books and writing code. What he didn’t realise is that technology cannot be “the best technology possible” in a vacuum. The question for a startup to answer is; the best technology for who? The ‘who’ in this equation is defined by two factors; people who want the technology; and are willing to pay enough for the technology such that the startup can build a profitable business. So the job of the lead engineer in a startup is not to just build excellent technology. They are part of a research team, and their main job is to operationalize the hypotheses of the startup as minimum viable products designed to maximise learning.

4.  A Significant Contribution

In science, a thesis is considered successful when the researcher makes a new and significant contribution to knowledge. For startups, there are similar pressures. A startup thesis is successful when it has discovered a profitable, sustainable and scalable business model. In a PhD thesis, you have to convince may be five to ten people, who constitute the toughest audience on earth, of the value of your work, before you can graduate. But customers are equally tough as an audience. In a democracy, you can’t make customer give you their money; they have to part with it voluntarily. So unless you are creating real value in their lives and solving real problems for them, your business is doomed to fail. So upon achieving product-market fit, a startup graduates.  

Entrepreneurship as a Managerial Science

Just like calling startups experiments, the thesis metaphor emphasises learning. The difference is that an experiment sounds like a single event, whereas it’s implicit in the terms thesis and dissertation that you are involved in a process that involves several research studies and iterations.  What I also like about the thesis metaphor is that it opens up entrepreneurs to learn more about the whole tool box of scientific methods that are available to them as they develop their idea. As I noted in my earlier post, the scientific method is not just experimentation. There are several other methods including surveys, interviews, customer observations, user testing and case studies. The thesis approach fully aligns startups with the scientific method and prepares ground for the development of a managerial science that can be taught as a learnable and repeatable process to entrepreneurs; just like scientists have been doing with PhD students for centuries. 

Is 'experiment' a good metaphor for startups?

I am Lean Startup fanatic. From the first time I read the material and heard Eric Ries speak it resonated with the geek in me and gave me insights that I have been using to help large organizations innovate like startups. I have also found that my background as a researcher and academic makes it easier for me to apply some of the tools such as customer development, cohort analysis and developing minimum viable products.

But something about the language that we use within Lean Startup has been bothering the scientist in me. It has been bothering me for a long time but more so lately, and I just can’t hold it in anymore:

An 'experiment' is NOT a good metaphor for describing startups…  

This issue has been raised before, but I think it may have been ignored. Defining a startup as an experiment does not really work in scientific terms. Firstly, because within your startup you will run a series of studies, of which only a few will be real experiments. Calling a startup an experiment makes it sound like a single event, when it’s more like an on-going series of studies, even after you achieve product-market fit.  

Secondly, most of what we do within Lean Startup just doesn’t meet the strict scientific standards that make the experiment the Holy Grail of scientific research. The experimental method is the Holy Grail because it is used to study cause and effect. After a researcher runs an experiment they should be able to say with some confidence that changes in variable X have a causal effect on changes in variable Y. To be able to reach such conclusions an experiment needs some of the following characteristics:

1. The ability to deliberately manipulate one variable (the independent variable), while holding other variables constant.
2. Some ability to deal with potential confounding variables that may affect your results. One way this is done within social science is to randomly allocate people to experimental conditions. Another way is to match research participants with regards to the variables you are trying to control.
3. The use of a control condition in order to take baseline measures upon which we will judge our results.

These rules/standards do not make experiments infallible. However, when an experiment is done properly following these rules gives researchers more confidence about the nature of the causal relationships between two or more variables. It also means that our experiments are replicable, which may be more important in proper science than in startups.

The Scientific Method
Looking at the above description, the only tool currently utilised within the Lean Startup that might meet the standards to be called experimentation is A/B testing, and this is only if the tests are properly designed to meet the above criteria. Cohort analysis may also be used as another powerful experimentation tool, especially if it is combined with A/B testing.

I think that the problem may lie in a confusion that views the scientific method as synonymous with experimentation. This is not necessarily the case. Experiments are only a sub-category of the scientific method. Developing falsifiable hypotheses is also an important part of the scientific method, but how you test those hypotheses is not necessarily with experimentation.

In Running Lean, which is a great book that I use in my own work, Ash Maurya defines an experiment as one full circle through the build-measure-learn loop. This is scientifically not correct. I think a successful circle through the build-measure-learn loop is better conceptualised as validated learning.  This is because you can go through the build-measure-learn loop several times, all the way to successfully achieving product-market fit without ever running a single experiment. However, you will run successful studies and you will learn a tonne about your customers’ problems and their perceptions of your proposed solution.

Indeed, Ash Maurya advises startups to validate qualitatively and verify quantitatively. This is wonderful advice that provides a practical actionable path through which entrepreneurs can take the right action at the right time. At the beginning, you really want strong signals from early adopters, so qualitative methods are appropriate. However, qualitative data collection is not usually associated with the experimental method. This does not make qualitative research non-scientific.  It is one of the tools within the arsenal of the scientific method. The same is true of all other research methods available to startups (e.g. case studies, interviews, surveys, user-testing, and customer observation). All these are powerful scientific tools that can provide valid data for startups to make informed decisions. But most of these cannot be described as experiments.

What Do We Want to Become?
The reason this issue has been bothering me is that I am concerned that the Lean Startup could grow into a pseudo-managerial-science that appears to use the scientific method and yet the practitioners do not understand scientific methods fully. This can create problems in terms of decision making. Each one of the research methods I cite above has its own limitations. If startups founders are unaware of these limitations they can make wrong decisions based on the data they collect. This can lead startups to use what they think are scientific methods and still fail.

For example, if your sample size is too small, and you didn’t use the right sampling methods, then you could pick up what you think is a strong signal from customers but is actually random noise. With small samples, the chances of this happening are actually quite high. This is why Ash Maurya encourages people to verify quantitatively the signals they get from qualitative research. But if people think their qualitative study is an experiment, with all the power associated with experimentation, they could feel more confident about their business ideas than they should. This could be particularly problematic for entrepreneurs who are already struggling with their reality distortion field.

In a recent post, Salim Virani raised an interesting question about how much the Lean Startup movement is learning from scientists and applying this learning to our work. I think we have choice during these early years of our movement. Do we want to fully align ourselves with scientific methodology and apply these methods appropriately to building startups? Or do we want to build our own approach with its own nomenclature and use methods that we don’t fully understand?  Is the full understanding of the scientific method really that important for startups? This is a choice we should make and be explicit about.

One approach is to say the Lean Startup method borrows from science a few things we find useful but our goal is not really to become a true management science for startups. I may be biased, but I think such an approach is just not good enough and will over the long term weaken the impact the Lean Startup method can have on entrepreneurship. This would be a real shame. Because until now there has been little opportunity to develop excellent methods for building startups that are teachable, replicable and scalable. I strongly believe the Lean Startup is just that method.  

Five Reasons Why You Can Develop a Product That 'Scratches Your Own Itch' and Still Fail.

"Scratch your own itch" is advice that you hear a lot in startup circles. According to this advise, when deciding what product to build you should build something that solves your own problems and worry about customers later. This view was made highly popular by the folks at 37Signals. However, this is a good example of entrepreneurs giving people advice on the basis of what worked for them. "Scratching your own itch" is not really a repeatable formula for startup success. If there are people who have succeed using this formula its probably mostly based on luck. Below are five reasons why you can build a product that scratches your own itch and still fail: 

1. How many people have that same itch: This goes to the potential size of your market. Just because you have an itch, does not mean that other people do as well. Do not fall into the false consensus trap that afflicts most people. 
2. How itchy is the itch for other people: This speaks to your ability to attract early adopters. You might think its a problem. But how bad is the itch for others. Is it just a minor annoyance or is it like eczema? If you are in the eczema situation, then the problem is so big that people are actively seeking solutions. If they are not, then no matter how well you scratch your own itch, your startup could still fail. 
3. How well does your solution scratch the itch: In other words, do you have product/market fit? The itch could be very strong among your customers. But if you build the product just for you, you will base it entirely on your own world-view and prior experiences. You cannot assume that people with a similar itch will be exactly like you. As such, you need to make sure that your customers find the product as useful and as easy to use as you assume it is. 
4. How much will it cost you to deliver the solution to customers:  Suppose you find early adopters, the job of an entrepreneur is not just build products. It is also important for you to figure out the costs of customer acquisition and  the costs of the channels you will use to deliver your solution to your customers. 
5. How much are the customers willing to pay for your solution: This piece, combined with number 4 above, will tell you whether or not you have a sustainable business model. If your customers are only willing to pay less than it costs you to make and deliver the solution, then your business will fail. And you can't make it up in volume! 

All the above reasons speak to the fact that a business is more than just its product. Scratching your own itch might be a good starting point, but you have to get beyond that to make sure that you are building something people want and also that you can build a sustainable business around your product. So when you decide to build a business around you own itch, don't go straight to building the product. Get out of the building and do some customer development. Only when you get strong signals around early adopters and a potentially good business model should you start building. 

Putting the Cart Before the Horse....

I love this image! This could be entrepreneurs and innovators in any organization. When doing innovation or launching a startup 'putting the cart before the horse' is when you build a product from start to finish, without ever establishing whether what you are building is what customers want...

Would I lie to you... Yes, I would!

When doing customer development, you need to make sure you are asking the right questions. In end, what most of us really want to know is whether customers would pay money for our product. If we ask customers vague questions about whether they 'like' our product, they will invariably say they do. Unless they have serious social skills deficits or your product smells like poo, most people will not tell you the truth about what they really think. The shock value of Simon Cowell on American Idol is that he actually tells you the truth.

Social desirability biases are a common problem in social science and market research. Most people will respond to direct questions by telling you what they think you want to hear, or by telling what they think will make them look good and impress you. This is the polite thing to do, right?

This problem is made worse if you then decide to do your research using focus groups. Now you have ratcheted up people's social desirability concerns by adding several other people for them to seek approval from. In that context it is hard to decipher signal from noise.

If the goal in a Lean Startup is to learn fast, then using some of these standard methods may not be useful. Within science, removing potentially confounding factors from our experiments is a central concern. Startup founders and organizations attempting to launch new products also need to learn to remove confounding factors from their experiments. Here are a few non-exhaustive pointers:

• Never ask people what they would do in particular situation, they have no idea. Its better to figure out a way to observe what they actually do. 
• Never ask people whether they would buy your product, ask them to buy it NOW. 
• Never ask people if they 'like' your product, unless your business model is winning popularity contests. This is literally a 'vanity metric'.  
• Never run a focus group, unless your product is "getting people to sit around talking with each other". 
• Never, ever, ever, ever talk about your solution, before you understand the customers' problem.
• Never, ever, ever, ever, ever do customer development research without an explicit hypothesis, and an idea of how you will know if you are right or wrong. 

Always, get out of the building and talk to customers. There are no answers in the building.... The challenge is making sure that you are doing it right.

I am in a cargo-cult, what do I do if the plane actually lands: Do entrepreneurs really learn?

In a recent paper, Frankish and colleagues reported research showing that entrepreneurs do not really learn, they just claim they do. They analysed data from 6854 firms and found no connection between previous business ownership and likelihood of future success in a new venture. People who have previously owned a business had the same failure rates as people who are just starting out. Frankish et al. attribute this finding to the strong role played by chance in entrepreneurial success (i.e. you can't really learn to play the lottery).

This is the interesting thing about entrepreneurship. You can do all the 'right things' and fail. You can also do all the 'wrong things' and succeed. You can also do all the 'right things' and succeed, but your success will have nothing to do with anything you were doing. If you read books on entrepreneurship and innovation, they are full of advice about what you should do or not do. Grow virally, charge from day one, do not charge from day one, raise money, don't raise money - bootstrap, built something that you find useful and others will to, build something beautiful, et cetera ad nauseum.

Most successful entrepreneurs provide advice based on their own experiences. However, no two entrepreneurship situations are exactly the same. This is why Frankish et al. found that previous entrepreneurial success does not predict future success. So following some entrepreneurship advice is a bit like being in a cargo-cult. You don't really know if the advice you are following is really relevant or useful for your situation. You are just doing the 'right things' because it is the 'right thing' to do...

But this is where it gets interesting. Imagine you are in a cargo-cult. You are doing all the actions you have seen other people do when planes were landing. You do it a few times and one day the plane actually lands! 

Whoa!!

If this ever happened, outsider observers might know that this is just a coincidence. You just happened to be doing your rituals when a distressed plane needed to land and was directed to your island.  However, for members of the cargo cult this will be certain confirmation that their rituals are the 'right thing' to do.

Now, in  real cargo cults there is no real chance of this happening or at least the chances are smaller than the likelihood of someone winning the lottery twenty times in a row in ten different countries. But in the cargo cults of entrepreneurial practice this can certainly happen, and it happens a lot! People's business ideas can succeed because of reasons other than what they are doing (e.g. having a Facebook page). However, people will in engage in illusory correlations and connect their actions to their success. This is a classic case of being fooled by randomness. This is also why some entrepreneurs are often disappointed/confused after they do all the 'right things' and their business fails.

The lean startup approach is powerful  because viewing the startup as an experiment in search of a sustainable business model stops entrepreneurs from going gung-ho on ideas that sound plausible but are not fully tested. And when done right, it also deals with the cargo-cult problem because entrepreneurs are forced to explicitly state hypotheses about what they expect to happen and to develop scientifically sound methods for testing these hypotheses.

Then we will know, with some level of certainty, that in this situation, for this particular startup, these actions really caused 'the plane to land'. We also know that in a different startup, in a different situation, even if it is working on the exact same product, those same actions may not necessarily lead to success.

Passion Plus Effort = Sweet-Spot

Recently, Mark Cuban wrote a post entitled Don't Follow You Passion, Follow Your Effort. In it he argues that people should not follow their passion, but rather follow their effort. He argues that the harder you work, the better you become at something and the more passionate you become about it. Interestingly, Cuban's advice to "Follow Your Effort" is a half-truth. It is a half-truth in the same way that "Follow Your Passion" is a half-truth. As with most things, reality is somewhere in the middle.

• Passion Without Effort is Just a Dream... We all know the guy who never gets off his backside and had the idea for E-Bay before E-bay! Just pursuing your passion/talent without realising the effort it takes to excel is just as likely to lead to failure. 

• Effort Without Passion is Drudgery... We all know some really hard working people who are excellent at their work, and yet at the same time really hate what they do for a living. For some its not by choice, sometimes you've got to do, what you've got to do. But this does not necessarily lead to passion developing. 

Within the Lean Startup framework, vision is equally as important as the everyday effort and tedium of innovation accounting (http://ow.ly/a9snD). Effort is applied in the service of vision. Furthermore, it is not just random effort, it is deliberate effort directed at validated learning. Otherwise people are just "busy doing nothing". Startups are hard work, and the only thing to keep you going when you are in the trough of sorrow, is commitment to your passion/vision, and a validated sense that you are on the right path. People also need to know that just because you are pursuing your passion does not mean that you don't have to work hard. If you think talent is enough, just check out Malcolm Gladwell's 10, 000 hour rule. Not many of us get the opportunity to pursue our passion and make money at the same time. But when we do, and we apply ourselves with deliberate effort, then we have found that sweet-spot!