Not All Failure is Smart Failure

Source:  Flckr

Several months ago, long-time executive and now Duke Professor Jon Fjeld published an article for Sloan Management Review titled, "How to Test Your Assumptions."   He begins the article by questioning the "fail fast" approach embraced by many entrepreneurs in recent years.   He notes that many startups do not understand the lean startup philosophy, and they are implementing it in a haphazard and ineffective way.  He writes: 

The enthusiasm surrounding the “lean startup methodology” and its many offshoots has created a mindset that entrepreneurs should just launch, failing early and often — iterating, to use startup parlance. But failure alone does not teach. If there are an infinite number of bad ideas, eliminating one gets us no closer to a good idea. Rather, the businessperson contemplating a new venture must begin by evaluating factors that have to be true for the venture to succeed. He or she also must model these factors in a way that allows for reasonable testing. For example, the assumption that people will buy a product for the asking price is a big one; it would take a full launch to completely validate this. Therefore, the entrepreneur must split big assumptions into discrete, manageable assumptions that can be tested at a level of detail allowing for efficient learning.

He makes a terrific point.  Not all failure is instructive.  At times, you can fail, but find yourself unable to distinguish among many causal factors to which that failure can be attributed.  In fact, these types of failures can be enormously costly, because individuals can derive incorrect attributions easiliy, and then make the wrong kinds of changes in hopes of averting failure in the future.  Failing fast works best when you can test your ideas in ways that enable you to identify what went wrong and WHY it went wrong.   Sometimes, two or three possible causes should be considered, and another test may be conducted to determine which of these factors led to the failure.  

The Downside of Testing, Prototyping, and Experimentation?

I'm a big fan of the concept of testing, prototyping and experimenting during the product development process.  I believe in David Kelley's philosophy: Fail often to succeed sooner.   I think that "test and learn" and rapid iteration beats formal planning every time in highly ambiguous environments.  However, new research suggests one very important limitation to testing that involves going to market with a minimal viable product (MVP).  

Andrea Contigiani, a researcher at the Mack Institute for Innovation Management and a Wharton doctoral candidate, assembled a fascinating dataset on the product development efforts at over 1,000 software startups. He looked at how these firms' approaches to product development changed after a landmark court decision, Alice Corp v. CLS Bank International, made patenting software less effective.  He wondered if firms changed their behavior after this court ruling, for fear that prototyping and testing might invite many imitators and therefore make it harder to establish and defend competitive advantage.  He explains his findings:

I found two things, broadly speaking. One is that after Alice took place, the affected companies changed strategies. In particular, they seem to be less likely to test their product; they do less experimentation. Given that they can no longer use patents to protect their ideas, experimentation becomes risky. Sure, you get the benefit of learning, but the cost of doing so goes up. So it’s not necessarily a good idea.

On the other hand, they seem to launch their product faster. They go to market sooner and so they can start getting feedback. Adapting your product, or pivoting, once you are in the market is a little harder [than early testing], because adaptation costs are higher. But on the other hand, once you do that you are essentially creating other barriers to imitation, like brand and network effects. So it is safer.

The second result is more about performance, and there I looked at what happened to companies that did a lot of experimentation while in the new post-Alice regime where they could not really protect through patents. I saw a negative correlation between doing that and performance.

Companies that experimented a lot without potential access to patent protection were less likely to get funding, and they also seemed less likely to get acquired. And so overall, this choice seemed to really affect the performance.

The findings are fascinating.  I would offer one important caveat though.  Contigiani is studying testing that involves actually going out into the market with a product, or at least exposing ideas to public inquiry and study.  Much testing, prototyping, and experimentation can take place in a very private way, so as to not enable potential rivals to learn about innovations.  It seems to me that firms ought not to be eliminating testing and prototyping, but thinking instead about how to do it in a way that protects their ideas from imitation.  

Pivot: The Road to Ruin for Many Startups

The word "pivot" has become part of the entrepreneurial lexicon over the past few years.   Eric Ries developed the "Lean Startup" methodology for building and launching new products.  Ries argues that entrepreneurs should work toward the development of an MVP - minimum viable product - and then listen carefully to customer feedback so that the next iteration of improvements can be put in place.  Entrepreneurs should use the minimum viable product as part of a process of disciplined experimentation, whereby they test core hypotheses about their business model.    As entrepreneurs gather feedback, they should pivot based on what they are learning.  According to Ries, a pivot is a "structured course correction designed to test a new fundamental hypothesis about the product, strategy, and engine of growth."

In a blog post for the Wall Street Journal, serial entrepreneur and Stanford Professor Steve Blank offers some cautionary words about the "pivot" concept though.  Blank gives an example of a founder who would rush back to his startup's offices after every customer visit and initiate a fire drill of sorts. The founder would constantly be advocating changes based on his most recent interaction with a customer.  Blank explains the problem:

Pivot as an Excuse
I wasn’t surprised when he pushed back: “I’m just getting out of the building and listening to customers. All I’m doing is pivoting based on their feedback.” By now I’ve heard this more times than I liked. “Yuri, one of the things that make you a great founder is that you have insight others don’t. But like all great founders, some of these insights are simply hallucinations. The problem is you and other founders want immediate action every time you have a new idea. That’s a mistake.
“A pivot is a substantive change to one or more of components to your business model.” You’re using “Pivot” as an excuse to skip the hard stuff – keeping focused on your initial vision and business model and integrating what you’ve heard if and only if you think it’s a substantive improvement to your current business model. There is no possible way you can garner enough information to pivot based on one customer’s feedback or even 20. You need to make sure it’s a better direction than the one you are already heading in.”

Nordstrom Innovation Lab

Eric Ries has written a terrific new book called The Lean Startup.  On his blog, he wrote recently about how his lean start-up principles can be applied in large corporations.  He profiled Nordstrom, with these two terrific videos about the Nordstrom Innovation Lab.  I recommend taking a look at his book and at these two videos.