Probabilities, Prediction Markets, and Popular Fallacies
With Hillary’s surprise victory over Obama in the New Hampshire primary, pundits everywhere are decrying the allegedly ‘wrong’ odds that prediction markets like Intrade were displaying prior to the announced results. (As just one example, Barry Ritholtz weighs in with his ‘explanation’ of : “Why Opinion Markets Fail“.)
At one point the betting markets were implying over a 90% probability for Obama to win. Does this mean they were ‘wrong’? No it does not. It is impossible to judge whether a given probability is/was correct based on the outcome of a single event.
A 90% probability simply implies that, if you encounter a series of events each with a 90% probability, then 9 times out of 10, the favored outcome will occur; and 1 time out of 10, the unfavored outcome will occur. Those like Ritholtz who are now calling the prediction markets ‘wrong’ are implying the following: if the probability is 90% for an outcome to occur, then that outcome should occur every time. In other words, if the odds are 90% in favor of something — it should happen 100% of the time! But this is obviously fallacious. If the outcome occurs 100% of the time, then the correct probability to assign to it would be 100% — not 90%.
To validly assess the accuracy of prediction markets, one needs to aggregate all the situations where the odds were 90%, and then calculate whether the favored outcome indeed occurred 90% of the time. (And do the same with each level of probability.) This — and only this — will tell you how accurate prediction markets tend to be.
Barry Ritholtz:
As every good prognosticator knows, if you couch your forecasts in probabilities, the innumeric will never know you were wrong. It’s a cheap trick for the easily fooled.
Imagine if instead of a “THE END IS NEAR” sign, every loon carried a sign that proclaimed:
THERE IS A 57% CHANCE THAT THE END IS NEAR!!!
The fact that this didn’t happen — and the 43% probability did — doesn’t mean this forecast was accurate. It merely meant that the person had proferred two possibilities and one of those two occurred. But the math remains unverified.
Neat trick: By your definition, PREDICTION MARKETS CAN NEVER BE WRONG, so long as they maintain a 1% possibility of the alternative outcome.
That’s hardly a satisfying defense…
