At VOX, David Hendry and James Reade examine the question, “How should we make economic forecasts?” Among the ideas discussed is whether prediction markets could be used to improve economic forecasting. Interesting suggestion and seeming to be worthy of additional exploration, but the authors don’t go too deep here. Instead, they assert that “prediction markets can be viewed as a form of … model averaging,” and then drift into a discussion of forecast averaging. I’m not sure that forecast averaging is a good way to look at prediction markets.
Here is what they say:
Prediction markets can be viewed as a form of forecast pooling or model averaging, a common forecast technique (Bates and Granger 1969, Hoeting et al 1999 and Stock and Watson 2004). That is, forecasts from different models are combined to produce a single forecast. In prediction markets, each market participant makes a forecast based on his or her own forecasting model, and the market price is some function of each of these individual forecasts.
Since the “prediction” implied by a prediction market is set by the marginal transaction, it depends not at all on the distribution of earlier trades, nor on the valuations of parties priced out of the market at the current price.
For example, consider two event markets: in the first 999 contracts trade at $0.50 and the 1000th and final trade is at $0.75; in the second 999 contracts trade at $0.76 and the 1000th and final trade is at $0.75. In the typical interpretation of prediction markets, the event is “predicted” to result with a 75 percent probability in both cases. However, averaging among the different predictions doesn’t get you that result.
(Well, strictly speaking the market price is “some function” of the prices – namely, one in which all trades but the last are weighted zero and the last trade is weighted one. You can call this “averaging,” but that isn’t the most useful explanation of the function.)
I’m not arguing that forecast averaging might not be a good idea in many situations, just that averaging doesn’t seem like a good way to explain what a prediction market is doing.
I think your description of the price/prediction of a PM is much too simplistic. When people are looking for a simple rule as to what a prediction market is saying, they often use the most recent trade or the current offered price. But whenever someone tries to do a test of a market and defines one of these as the outcome they quickly decide that that number is too easy for the participants to game, and they come up with something else. I’ve heard the average over some period before closing, a randomly chosen price from some predetermined period and other ideas.
The best description I’ve seen of how to interpret the actual outcome was in some experiments done by Dave Porter, and Ryan Oprea, in which they had non-trading judges watch the ticker and estimate the actual value of the goods. The real answer is that the meaning of a market is how an observer would interpret it. In your first case, if all the trades at .50 happened a week before closing, and the only recent trade was at .75, that means something very different from if all the .50 trades were spread evenly over a long period, and there was one trade at .75 just before closing. In the latter case, many observers would say the market really believed the price was .50, and someone had snuck in a wild trade at the end.
Of course, in both these examples, I’m agreeing with Giberson that Hendry and Reade are way off base in describing PMs as an averaging process.
Your technical point is helpful. Prediction markets are a more sophisticated tool than an averaging mechanism.
However, they say, “can be viewed as”… vague enough… That is not untrue if you consider the outcomes of each of these 2 collective forecasting techniques. They are very close. The improvement brought by prediction markets is quite small (if any) —ask Yahoo! Research’s Daniel Reeves.
So I would agree with their language: …”can be viewed as”… The outcomes are very close… even though the aggregation information mechanisms are different.
“A prediction market populated by *toddlers* will not forecast the next UK election winner simply because it is a prediction market.”
http://www.voxeu.org/index.php?q=node/3647
Why do they put “toddlers” into that???
Traders are not toddlers. Why *toddlers*?
Are they being mean towards the traders or do they have a bigger picture to convey in their conclusion that I missed?
“Hendry and Reade are way off base”
They are “way off base” in theory but not in practice. Wait till Daniel Reeves’ paper comes out.
Michael wrote:
“Since the “prediction†implied by a prediction market is set by the marginal transaction, it depends not at all on the distribution of earlier trades, nor on the valuations of parties priced out of the market at the current price.”
This is true for some markets, but not all. Consider a winner-take-all market, similar to many on Hubdub. These PMs function more like parimutuel betting. The most recent bet on a state is added to all previous bets, readjusting the odds for all states. This is much different than the case where the most recent trade sets the market price.
Perhaps the best way to get the market prediction is to obtain the mean from the distribution of outstanding shares when the market closes. You can also obtain the distribution for use as a measure of uncertainty.
It is difficult to imagine how a prediction market to predict an economic forecast might work. It could work if the outcome were known with a reasonable degree of certainty, but that isn’t the case, usually. When I see economic forecasts showing a 1.2% contraction in 2009 and 2.4% growth in 2010, I laugh. Such forecasts are politically influenced so as not to scare the wits out of the citizenry. All economic statistics are really *estimates* of what the economists think is happening (or has just happened). Often these figures get revised when additional data comes in, well after the fact.
Who’s to say which figure should be used for the payoff. It could be that the prediction market is simply trying to predict the future estimate. Hardly valuable. To be useful, the predictions need to be made well in advance, but most participants will not be very interested in these long-term decisions.
Finally, who should be invited to participate? Joe six-pack? Economists? Politicians?
“Finally, who should be invited to participate?”
Any information aggregator.
Any information aggregator could set up the markets. I was getting at who would participate in the trading. You probably don’t want too many American Idol bettors trying to forecast future economic conditions, unless you have a few knowledgeable economists to take advantage of them. I think you would need to have a *very* large betting pool, with as much diversity as possible.
Paul, you seem to be thinking of beauty contest markets, where the returns to the traders are set by the last trading price. This kind of market has a very different incentive structure for the bettors/investors than a prediction market, where the return to the traders is set by the actual outcome to be predicted.
You’re right that it’s harder to get significant participation in PMs that concern distant events, but it can be done. And if you don’t do it, you uncouple the betting from outcomes, which will of course make the predictions much less useful.
Part of the trick in designing useful markets is to come up with outcomes that can be determined for certain at some point. So the level of the S&P will be known for sure, but unemployment isn’t final until sometime after the date concerned. FX usually does pretty well. HSX has something that can be predicted, IEM doesn’t have any problems (though rarely, the market can’t be closed on time due to challenges of results.) You just have to find something where an answer is usually available close to the closing time.
Thanks for the reactions.
Chris M., I considered the possibility that the authors were using the term “averaging” in sort of a casual way to mean “some sort of tool for combining information.” The also say things like “forecast pooling” and “some function of each individual forecast” which doesn’t commit them explicitly to averaging as the method. But immediately after those more general references they jump into an explicit discussion of the problems of averaging, so I conclude that averaging is their focus.
Chris H., the Porter and Oprea reference is interesting, I just vaguely remember the paper and will have to go back and read it. But we can distinguish between how, behaviorally, people interpret market price sequences and the relation between market price sequences and event outcomes. Both halves of this distinction are potentially interesting.
Paul, you are right that I was thinking of a double auction style prediction market, and there are other market designs also of interest.
On the question of ‘who should be invited to participate,’ my inclination is to want to take as many people’s money as possible, so let everyone in. You might have intended the question more analytically – companies with internal prediction markets presumably have to address this question and it may make a difference in outcomes. I don’t know whether this issue is addressed in published studies.