The Downside of Testing, Prototyping, and Experimentation?

I'm a big fan of the concept of testing, prototyping and experimenting during the product development process.  I believe in David Kelley's philosophy: Fail often to succeed sooner.   I think that "test and learn" and rapid iteration beats formal planning every time in highly ambiguous environments.  However, new research suggests one very important limitation to testing that involves going to market with a minimal viable product (MVP).  

Andrea Contigiani, a researcher at the Mack Institute for Innovation Management and a Wharton doctoral candidate, assembled a fascinating dataset on the product development efforts at over 1,000 software startups. He looked at how these firms' approaches to product development changed after a landmark court decision, Alice Corp v. CLS Bank International, made patenting software less effective.  He wondered if firms changed their behavior after this court ruling, for fear that prototyping and testing might invite many imitators and therefore make it harder to establish and defend competitive advantage.  He explains his findings:

I found two things, broadly speaking. One is that after Alice took place, the affected companies changed strategies. In particular, they seem to be less likely to test their product; they do less experimentation. Given that they can no longer use patents to protect their ideas, experimentation becomes risky. Sure, you get the benefit of learning, but the cost of doing so goes up. So it’s not necessarily a good idea.

On the other hand, they seem to launch their product faster. They go to market sooner and so they can start getting feedback. Adapting your product, or pivoting, once you are in the market is a little harder [than early testing], because adaptation costs are higher. But on the other hand, once you do that you are essentially creating other barriers to imitation, like brand and network effects. So it is safer.

The second result is more about performance, and there I looked at what happened to companies that did a lot of experimentation while in the new post-Alice regime where they could not really protect through patents. I saw a negative correlation between doing that and performance.

Companies that experimented a lot without potential access to patent protection were less likely to get funding, and they also seemed less likely to get acquired. And so overall, this choice seemed to really affect the performance.

The findings are fascinating.  I would offer one important caveat though.  Contigiani is studying testing that involves actually going out into the market with a product, or at least exposing ideas to public inquiry and study.  Much testing, prototyping, and experimentation can take place in a very private way, so as to not enable potential rivals to learn about innovations.  It seems to me that firms ought not to be eliminating testing and prototyping, but thinking instead about how to do it in a way that protects their ideas from imitation.