Damped polls outperform prediction markets.

No Gravatar

Forecasting Principles:

Damping polls

Evidence from the literature shows that polls, in particular early in the campaign, are not reliable in predicting election outcomes but tend to overestimate the extent to which a candidate leads. To deal with these uncertainties, we added a damping factor to the RCP poll average. Damping is used to make forecasts more conservative in situations involving high uncertainty.

Our poll damping is based on research conducted by Campbell (1996) who showed that polls have to be discounted in order to achieve more reliable forecasts. Performing a regression analysis on historical poll data for the elections from 1948 to 2004, he derived a formula for discounting the polls according to their distance from Election Day. Campbell provided Polly the formula, along with a list of damping factors that vary by the number of days left before the election.

Currently, polls are discounted with a damping factor of ${factor}. Applying this factor, we calculate Polly&#8217-s discounted poll based forecast thus:

Polly&#8217-s poll based forecast = ((Latest RCP polling average – 50) * (1 – [damping factor])) + 50 = ((46.1 – 50) * (1 – 0.17)) + 50 = 46.8

Latest RCP polling average 46.1
Damping factor 0.17
Polly&#8217-s poll based forecast 46.8

Thus, our poll damping discounts a candidate&#8217-s lead in the two-party vote, depending on the days left prior to election. The further away the election day, the larger the damping.

Such damped polls have been shown to outperform sophisticated forecasting approaches like prediction markets. Comparing damped polls to forecasts of the Iowa Electronic Markets, Erikson and Wlezien (2008) showed that the damped polls outperformed both the winner-take-all and the vote-share markets.

Thanks to Andreas Graefe for the link.

Are Political Markets Really Superior to Polls as Election Predictors? – PDF file

For now, our results suggest the need for much more caution and less naive cheerleading about election markets on the part of prediction market advocates.

Previously: The truth about prediction markets

4 thoughts on “Damped polls outperform prediction markets.

  1. David Pennock said:

    In the title, do you mean “Damped polls”?

  2. Panos Ipeirotis said:

    Interestingly enough, if this paper is correct, then this suggests an obvious trading strategy for the traders in future election markets. So, in a sense, the markets would incorporate the information in this article and produce more accurate results next time :-)

    Now, being critical: Being accurate in out-of-sample data just 4/5 times is not really impressive. Is someone wanted to be really a PITA, the result does not even suggest statistical significance using a sign test, as there is 18% probability that the result is due to pure chance. Now, being more realistic: they did not have that many elections to work with.

  3. Chris F. Masse said:

    Panos Ipeirotis: Yes, that is what Justin Wolfers wrote in a comment on a blog post from the Andrew Gelman blog, which I already linked to months ago.

  4. Chris F. Masse said:

    David Pennock: Sorry for the typo.

Leave a Reply to Panos Ipeirotis Cancel reply

Your email address will not be published. Required fields are marked *