Testing Your Assumptions

You've used AI and/or Excel to build a beautiful model that examines the ROI of a new venture, project, or initiative.  The result? A strong case for investment, because the ROI is quite good.  You are ready to make your case to senior executives and/or investors.  How can you stress test your model?  The most important thing you can do is to test your assumptions. After all, garbage in, garbage out. If you build a model based on false assumptions, you are in big trouble.  Of course, confirmation bias afflicts many of us as we make decisions. We may be picking and choosing assumptions to deliver a rosy picture, rather than making more realistic presumptions about the future.  

How do we decide which assumptions warrant the most attention and should be tested vigorously?  Jon Fjeld was a long-time tech industry executive, and he now teaches at Duke's Fuqua School of Business.  He has a simple method for determining which assumptions need to be scrutinized first.  He argues that three factors are critical:

1. Severity:  How big is the impact on the project if the assumption is not true?

2. Probability: How likely is it that the assumption is not true?

3. Cost:  How expensive and time consuming is it to test the assumption?

Fjeld uses these three factors to create a simple ratio that can be used to rank your assumptions.  His equation is: (Severity x Probability)/Cost.   The higher the ratio, the more important it is that your prioritize the testing and validation of that assumption.  While we don't actually have a clear way to quantify these three factors, the concept of this ratio makes good sense.  By thinking about these three factors, and their relationship to one another, we can do a better job of deciding how we want to test, experiment, and prototype before making a big bet.