How You Think vs. What You Know

Philip Tetlock wrote a terrific book last year titled Superforecasters.  In that book, he describes the  decades of research he has conducted on expert predictions. Put simply, he has found that experts don't make great prognosticators, despite their vast knowledge and experience. Knowing a great deal does not enable you to see the future. In fact, it can be a handicap at times, because you may be tied down by conventional wisdom, long-held assumptions, and pre-existing beliefs. In Tetlock's research, he has found that a small set of people have an uncanny ability to make accurate predictions. These people are not experts in the domain in which they made those forecasts. What enables them to predict so effectively? Tetlock conclues that the key is HOW THEY THINK, not what they know. The best forecasters exhibit the following characteristics:

• They are open-minded, reflective, and intellectually curious. 
• They acknowledge what they do not know. 
• They gather information from a wide variety of sources and question the validity of each source. 
• They enjoy pondering a range of diverse views, and they update their conclusions as facts change. 
• They treat beliefs as testable hypotheses rather than hard truths.

In my view, the same might be said about the best leaders.  They often distinguish themselves by how they think, not what they know.   The best leaders are reflective and intellectually curious, question the conventional wisdom, do not become wedding to existiing beliefs, admit what they do not know, critique the validity of information sources, and gather diverse opinions before making key decisions.  

More on Making Good Predictions: An Interview with Philip Tetlock

Superforecasters: Foxes vs. Hedgehogs

You can't go anywhere, watch anything, or read any article these days without hearing from an expert or pundit predicting the future.  Should we pay much attention to those forecasters?  Well, according to research by Philip Tetlock and his fellow scholars at the Good Judgment Project, we might want to remain skeptical of such predictions.   They have found that many experts are not very accurate in their predictions.  On other hand, they discovered some folks who they term "superforecasters."  Who are these folks?  Here's an excerpt from a BBC story on Tetlock's research:

Roberts cites the Isaiah Berlin essay “The Fox and the Hedgehog” – a comparison often used by Tetlock himself – which divides thinkers into those “hedgehogs'” narrowly invested in a single topic and “foxes” with a wider, if shallower, range of experience. “Foxes” like him, Robert says, tend to be better forecasters. “They don't get attached to one particular narrative” and are able to adapt their viewpoints to incorporate any new information, unlike “hedgehog” thinkers, who often force new information into a pre-existing mental framework, or discard it if it seems to contradict their initial view.

In short, the problem with many experts is that they are hedgehogs.  They have deep expertise in a narrow domain, but as a result, they also have a lot of biases.   They have a strong attachment to certain pre-existing beliefs.  The foxes are more open-minded.  What else characterizes the superforecasters?

1.  An ability to ask good questions

2.  An ability to assimilate new information, even if it contradicts their initial view

3.  An ability to use the scientific method to pose and test hypotheses

4.  A reliance on data and not just intuition

5.  A willingness to acknowledge their own lack of knowledge in certain areas