PDF file.

I don’t think there is any question that the usefulness of prediction markets has been exaggerated. We don’t hear much from James Surowiecki anymore. So a few of those polled probably indicated that he doesn’t exaggerate the usefulness (at least not anymore). Justin Wolfers isn’t as well known as Robin Hanson in the prediction market arena. This probably accounts for the large size of the “Don’t Know” vote. This leaves the “Yes” category, which would be made up of Robin Hansonites, who believe the slightest possible improvement in predictive accuracy over other methods justifies a significant investment in prediction markets.

If the question had been asked in a prediction market instead of in a poll, such answers would have led us to believe that the market knows little about the subject, if anything at all. Of course it would have been difficult, if not impossible, to create a prediction market in which the “outcome” is an opinion!

-

UPDATE: Here’s an example (among many) of the grave problems with prediction markets, which are unreported by the media.

-

PDF document.

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

In part because of disappointing findings such as this, an increasingly popular substitute for expert opinions are so-called “prediction markets,” in which individuals buy and sell contracts on various outcomes, such as football game point spreads or presidential elections. The market prices for these contracts then effectively aggregate the knowledge and judgment of the many into a single prediction, which often turns out to be more accurate than all but the best individual guesses.

But even if these markets do perform better than experts, they don’t necessarily do a good enough job to rely on. Recently, my colleagues have started tracking the performance of one popular prediction market, at forecasting the outcome of weekly NFL games. So far, what they’re finding is that the market predictions are better than the simple rule of always betting on the home team, but only slightly so — which, oddly, is very similar to what Tetlock found regarding his experts. Some outcomes, in other words, and possibly the outcomes we care about the most, simply aren’t “predictable” in the way we would like.

-

1. Prediction markets are not “a substitute for expert opinions”. They are a substitute for the averaged probabilistic predictions of a large group of experts polled the traditional way (by phone or by e-mail). In prediction markets, traders (who are not experts, most of the times) collect and aggregate facts and expertise at a lower cost than a poll or survey of experts.
2. In the research cited by Ducan Watts, the prediction markets are slightly more accurate than the competitive forecasting mechanism. Well, that’s something we are used to.
3. What Ducan Watts doesn’t say is that prediction markets integrate facts and expertise faster than the group of experts polled by his researching colleagues —for the very crude reason that it takes a certain time to survey a group of experts (be it by e-mail or by phone).

-

If I can count, that’s 3 reasons why prediction markets can bring in business value:

1. lower cost;
2. better accuracy (relatively, and, overall);
3. velocity.

That said, it should be repeated that prediction markets feed on facts and expertise —so the experts remain indispensable in the general forecasting process.

No facts (e.g., political polls) –> No prediction markets.

No experts (e.g., NFL prognosticators) –> No prediction markets.

-

Video

-

-

Predictify is not a prediction exchange. We think prediction markets are superior to polls and surveys, don’t we?

With Predictify, the mechanism delivering the collective verdict is simplistic: it’s a poll —with possibility to get down to each individual answer.

Their conversation is very interesting, nevertheless —in great part due to Robert Scoble’s intense curiosity.

Technically, the video is awesome and plays well —even with my old computer and slow DSL line. Kudos to the Fast Company techies.

UPDATE: I don’t like that their video starts off automatically, though. With YouTube, we decide to play the video —it is not imposed on us.

UPDATE: Alas, their embedded video does not go into the blog feed.

-

UPDATE: I e-mailed my remarks to Robert Scoble, and he’s asked to his techie to look into the issues.

-

UPDATE: I see that the video does not start on its own, now. They managed to correct that. Rest the fact that their videos don’t go into feeds.

-