Philip Tetlock interviewed (last year):
IDEAS: Any individual expert is likely to be wrong. What happens when you put a bunch of them in the same room? Does the consensus come up with a better prediction than the individual?
TETLOCK: Usually yes. Jim Surowiecki wrote that very clever book on “The Wisdom of Crowds.” . . . Sometimes social scientists call it the miracle of aggregation. It’s an interesting fact that the average of the experts’ predictions often outperforms the individuals from whom the average was derived.
IDEAS: But the consensus can be wrong, too, as in the case of the widely shared view that Iraq had weapons of mass destruction. How does that happen?
TETLOCK: The crowd can certainly be wrong. Sometimes, the crowd can be dramatically wrong. . . . One big danger is if you’re dealing with an ideologically, intellectually homogeneous group, and they’re all talking to each other. They can really talk each other into some quite extreme positions.
-
Philip Tetlock interviewed (this year):
What makes some forecasters better than others?
The most important factor was not how much education or experience the experts had but how they thought. You know the famous line that [philosopher] Isaiah Berlin borrowed from a Greek poet, “The fox knows many things, but the hedgehog knows one big thing”? The better forecasters were like Berlin’s foxes: self-critical, eclectic thinkers who were willing to update their beliefs when faced with contrary evidence, were doubtful of grand schemes and were rather modest about their predictive ability. The less successful forecasters were like hedgehogs: They tended to have one big, beautiful idea that they loved to stretch, sometimes to the breaking point. They tended to be articulate and very persuasive as to why their idea explained everything. The media often love hedgehogs.
How do you know whether a talking head is a fox or a hedgehog?
Count how often they press the brakes on trains of thought. Foxes often qualify their arguments with “however” and “perhaps,” while hedgehogs build up momentum with “moreover” and “all the more so.” Foxes are not as entertaining as hedgehogs. But enduring a little tedium is worth it if you want realistic odds on possible futures.
So if you were looking for a money manager, you’d want a fox?
If you want good, stable long-term performance, you’re better off with the fox. If you’re up for a real roller-coaster ride, which might make you fabulously wealthy or leave you broke, go hedgehog.
But it was doomster hedgehogs like money managers Robert Rodriguez and Jeremy Grantham who first saw the crisis coming.
Hedgehogs are sometimes way, way out front. But they can also be way, way off.
-
His book: Expert Political Judgment: How Good Is It? How Can We Know?
