In a new study, Daniel Reeves, Duncan Watts, Dave Pennock and I compare the performance of prediction markets to conventional means of forecasting, namely polls and statistical models. Examining thousands of sporting and movie events, we find that the relative advantage of prediction markets is remarkably small. [...]
Given that sports and entertainment markets are among the most mature and successful, our results challenge the view that prediction markets are substantively superior to alternative forecasting mechanisms. Nevertheless, it is certainly possible that there are forecasting applications where either the relative advantage of markets is larger, or that such differences in performance are consequential. Thus, while prediction markets may yet prove to be useful, it would seem the enthusiasm for their predictive prowess has outpaced the evidence.
Prediction Without Markets — by Sharad Goel, Daniel M. Reeves, Duncan J. Watts, David M. Pennock
http://www.cam.cornell.edu/~sharad/papers/pred-wo-markets.pdf
-
Robin Hanson’s comment:
Whoever said that every prediction market would always be more accurate than any other mechanism? [*] I’d say they are more-accurate more often than they are less-accurate, compared to mechanisms with similar resources. And your plots at the bottom look like they are testing calibration, not accuracy.
Read Daniel Reeves’s answer to Robin Hanson.
[*] Google, doc.
