Midas Oracle .ORG

Chris F. Masse June 9th, 2008

Justin Wolfers (in January 2008):

1. Bob and Chris has four elections in their data, so it is hard to draw too much from it. That said, I draw two conclusions. First, markets beat an unconditional use of polls as forecasts. Second, correcting the polls, the two are pretty darn close. Based on a sample of four elections, I’m not sure I am willing to call one method or the other the winner.

2. My conclusion is that this actually tells us a lot of what prediction markets do: they digest and aggregate the polls, and create a pretty useful adjusted forecast. If I don’t have the time to do the careful aggregation that Bob and Chris have, then I’m glad to have the market to do this for me. So I interpret their paper as telling me something about the mechanism by which prediction markets do well in forecasting elections. (This suggests an interesting puzzle: why did prediction markets do well in the pre-polling era?)

3. Your comment that polls are a snapshot, not a forecast, strikes me as a bit beside the point. Many people use polls as a forecast, and so it is reasonable to ask if they are a good forecast. (And pollsters sell them as forecasts, until you suggest they don’t do well, and then they fall back on the “snapshot” argument.)

4. Bob and I have a friendly bet (I can’t remember whether it’s a bottle of wine, or not), on whether his results will hold up in a larger sample. Given that we have both state-by-state polls and state-by-state markets for the 2004 and 2008 election cycles, one can extend his approach so that we get around 100 markets (rather than four). We are planning to work together on this. My guess is that this larger sample will confirm the view that markets beat even Erikson-adjusted polls, although Bob’s guess is the opposite.

Point is, the facts can resolve our debate, and we are planning on doing exactly that. I think it will be fun to work with Bob on this, as he is clearly much more deeply embedded in many of the issues around political forecasting than I am. It should be a fun learning opportunity. [...]

-

I think that the strongest evidence for prediction market efficiency comes from sports betting markets. There’s a ton of data out there, and while there are some smallish anomalies (including the favorite-longshot bias), the evidence in favor of the performance of markets v. experts is really very strong there. As you know, I grew up working for bookies, so this really was where I began. My colleagues in finance would suggest that the real evidence comes from equity / bond / futures / options markets, but there are enough well-known anomalies in those domains that I’m a bit less convinced. (And if you were making the rhetorical point that prediction market aficionados often overstate the evidence for their beliefs, I completely agree.)

I hadn’t really coded your piece with Chris as being driven by the favorite-longshot bias, but now you say it, the point is obvious. I’ll make sure to reference your paper in the draft that Eric and I are working on about favorite-longshot biases in political markets generally. (I think you saw early work on this, while in Palm Desert.) For Andrew’s benefit: we have collected data on literally hundreds of political prediction markets, and find fairly pervasive evidence of a favorite-longshot bias. The work on that paper isn’t done yet, but the message is very unlikely to change. There is a very strong favorite-longshot bias in political prediction markets.

BTW, I’m willing to make an even stronger forecast than Bob: I think the unadjusted markets will outperform even the Erikson-adjusted polls. (And for sure, the Wolfers-adjusted markets will beat the Erikson-adjusted polls.) And yes, I’m partly informed on that score by my experience with sports betting markets.

In terms of your question as to whether markets include info not in polls, I think you are right to infer that a lot of what markets are doing is following polls. Equally, the strongest evidence of markets doing some independent info aggregation comes from looking at political prediction markets in the pre-polling era. (If you haven’t seen it, I think you will really enjoy Rhode and Strumpf’s paper in the Journal of Economic Perspectives, where they review political prediction markets back to 1880.)

Also, I think that there are some pretty clear examples of the markets getting ahead of polls. Take Fred Thompson, who still does moderately well in national polls, but who the markets have written off. Obviously the polling result is driven by pure name recognition, and so what the markets are doing here is appropriately discounting the predictable polling errors, but they are using non-poll info to do this. (eg They don’t think Hillary’s advantage will also dissipate.) [...]

-

Are Political Markets Really Superior to Polls as Election Predictors? - (draft: PDF file) - by Robert S. Erikson and Christopher Wlezien - 2008-05-02

MO excerpts

-

Tags: accuracy, event deivative markets, event derivative markets, event derivatives, Iowa Electronic Markets, Politics, polls, prediction markets, relative accuracy

• All Best Posts Ever , Analysis (Data) - Prediction Accuracy (Absolute / Relative) & Prediction Precision , Exchanges & Markets , Forecasting
• Comments(6)