Abstract

Election markets have been praised for their ability to forecast election outcomes, and to forecast better than trial-heat polls. This paper challenges that optimistic assessment of election markets, based on an analysis of Iowa Electronic Market (IEM) data from presidential elections between 1988 and 2004. We argue that it is inappropriate to naively compare market forecasts of an election outcome with exact poll results on the day prices are recorded, that is, market prices reflect forecasts of what will happen on Election Day whereas trial-heat polls register preferences on the day of the poll. We then show that when poll leads are properly discounted, poll-based forecasts outperform vote-share market prices. Moreover, we show that win-projections based on the polls dominate prices from winner-take-all markets. Traders in these markets generally see more uncertainty ahead in the campaign than the polling numbers warrant—in effect, they overestimate the role of election campaigns. Reasons for the performance of the IEM election markets are considered in concluding sections.

Conclusion

This paper has tested the claim that the Iowa Electronic Market offers superior predictions of election outcomes than the snapshots from public opinion polls. By our tests, the IEM election markets are not better than trial-heat polls for predicting elections. In fact, by a reasonable as opposed to naïve reading of the polls, the polls dominate the markets as an election forecaster. This is true in the sense that a trader in the market can readily profit by “buying” candidates who, according to informed readings of the polls, are undervalued. Moreover, we find that market prices contain little information of value for forecasting beyond the information already available in the polls. Where then do the markets go wrong? To begin with, consider the vote-share market. The histories of market prices show that traders tend to hold persistent beliefs about the vote division that contradict the polls and that these persistent beliefs are often wrong. Incorrect beliefs get corrected only in the last days before the election, when the polls are difficult to ignore. The winner-take-all market tracks the vote-share market but compounds its errors by overvaluing long-shot candidates’ chances of victory, as if the market expects more campaign surprises than occur in reality. The existence of persistent mistakes in the vote-share market compounded by the degree of uncertainty about the vote-share estimates makes the winner-take-all market a particularly poor forecasting tool. Based on the experience of the IEM, if the polls show a candidate to hold a decisive lead but the market is unconvinced, bet on the polls. It should be noted that our daily poll projections are themselves rather crude instruments. Our robotic trading programs are informed by a flat prior, relying solely on the current polls and the days until the election but nothing more. Even when we compare market prices to the weekly average of poll-based forecasts, our instrument is primitive in that the week’s polls are not weighted for relative recency. But further perfection of our forecasting model from the polls would only advance our central argument. If we were to apply more rigorous modeling to obtain a properly weighted average of current polls and earlier polls, the victory of poll forecasts over the market forecast presumably would be more secure. One could argue that the results are drawn from a limited number of election years from a toy market with thin volume and limits on trader spending. With time, the IEM record could improve, and there is some suggestion that it has. Full-blown markets like Tradesports.com [or InTrade.com or BetFair.com] might in the end achieve an efficiency that so far has eluded the Iowa Electronic Market. Additionally, studies like the present one can suggest improved strategies to traders, which in turn improve the efficiency of election markets. Since our results are confined to a few runs of the toy Iowa market, some might claim a “so what” reaction. To such claimants, an important reminder is that the allegedly uncanny performance of the Iowa market has been touted as the primary evidence for the supposed superiority of election markets over the polls as an information source. The Iowa election market’s performance has not been so special after all. For now, our results suggest the need for much more caution and less naïve cheerleading about election markets on the part of prediction market advocates.

Damping polls

Evidence from the literature shows that polls, in particular early in the campaign, are not reliable in predicting election outcomes but tend to overestimate the extent to which a candidate leads. To deal with these uncertainties, we added a damping factor to the RCP poll average. Damping is used to make forecasts more conservative in situations involving high uncertainty.

Our poll damping is based on research conducted by Campbell (1996) who showed that polls have to be discounted in order to achieve more reliable forecasts. Performing a regression analysis on historical poll data for the elections from 1948 to 2004, he derived a formula for discounting the polls according to their distance from Election Day. Campbell provided Polly the formula, along with a list of damping factors that vary by the number of days left before the election.

Currently, polls are discounted with a damping factor of ${factor}. Applying this factor, we calculate Polly’s discounted poll based forecast thus:

Polly’s poll based forecast = ((Latest RCP polling average – 50) * (1 – [damping factor])) + 50 = ((46.1 – 50) * (1 – 0.17)) + 50 = 46.8

Latest RCP polling average 46.1
Damping factor 0.17
Polly’s poll based forecast 46.8

Thus, our poll damping discounts a candidate’s lead in the two-party vote, depending on the days left prior to election. The further away the election day, the larger the damping.

Such damped polls have been shown to outperform sophisticated forecasting approaches like prediction markets. Comparing damped polls to forecasts of the Iowa Electronic Markets, Erikson and Wlezien (2008) showed that the damped polls outperformed both the winner-take-all and the vote-share markets.

Thanks to Andreas Graefe for the link.

-

Are Political Markets Really Superior to Polls as Election Predictors? – PDF file

For now, our results suggest the need for much more caution and less naïve cheerleading about election markets on the part of prediction market advocates.

Previously: The truth about prediction markets

-

The quality of presenters is excellent and the workshop has succeeded in attracting high level contributions. The program and complete list of papers to be presented are available here, while directly related to prediction markets papers are:

• Are Political Markets Really Superior to Polls as Election Predictors? by Bob Erikson, Columbia University and Chris Wlezien, Temple University
• Optimal Price Setting in Fixed-Odds Betting Markets Under Information by Raphael Markellos and Vasiliki A. Makropoulou, Athens University of Economics and Business
• Hitting Home Runs and the Art of Corporate Valuation: Do Managers or the Prediction Market Make Better Predictions? by Orly Sade, New York University and Hebrew University
• Prediction Markets as a Tool for Management of Political Risk by Ole Jakob Bergfjord, Norwegian School of Economics
• Public Signal Bias and Prediction Market Accuracy by Tom Gruca and Joyce E. Berg, University of Iowa
• Ignorance Prior Bias in Prediction Markets by Lionel Page, University of Westminster
• An Analysis of Tradesports’ 2005-06 National Football League Prediction Market by Feng Zhou and Philip O’Connor, University of Waikato
• Testing the Efficiency of Markets in the 2002 World Cup by Steve Levitt, University of Chicago and Ricard Gil, University of California at Santa Cruz
• Price Formation in Sports Prediction Markets: A Cross-Cultural Study by Stefan Luckner, University of Karlsruhe
• Socially Embedded Prediction Markets by Yiling Chen and David M. Pennock, Yahoo! Research
• Manipulators Increase Information Market Accuracy by Robin Hanson, George Mason University and Ryan Oprea, University of California, Santa Cruz
• Is There a Favorite-Longshot Bias in Election Prediction Markets? by Justin Wolfers, University of Pennsylvania, Eric Zitzewitz, Stanford University and Andrew Leigh, Australian National University
• Using Markets to Inform Policy: The Case of the Iraq War by Eric Zitzewitz, Stanford University and Justin Wolfers, University of Pennsylvania

The exponential pattern of growth recorded in my literature review of the field remains consistent, at least.

Cross-posted from gtziralis.com.