I don’t think there is any question that the usefulness of prediction markets has been exaggerated. We don’t hear much from James Surowiecki anymore. So a few of those polled probably indicated that he doesn’t exaggerate the usefulness (at least not anymore). Justin Wolfers isn’t as well known as Robin Hanson in the prediction market arena. This probably accounts for the large size of the “Don’t Know” vote. This leaves the “Yes” category, which would be made up of Robin Hansonites, who believe the slightest possible improvement in predictive accuracy over other methods justifies a significant investment in prediction markets.

If the question had been asked in a prediction market instead of in a poll, such answers would have led us to believe that the market knows little about the subject, if anything at all. Of course it would have been difficult, if not impossible, to create a prediction market in which the “outcome” is an opinion!

-

UPDATE: Here’s an example (among many) of the grave problems with prediction markets, which are unreported by the media.

-

PDF document.

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

UPDATE: The poll result.

How do we know, now, that Intrade’s market price was not an accurate estimate of the probability bin Laden was killed or captured by September 2011? Is an prior estimate of 50 percent likelihood that a tossed coin will come up heads wrong if the coin comes up as “100 percent” heads (and not half-heads and half-tails)?

I’m not buying Chris’s implied definition of success and failure.

However, one might ask Robin Hanson about what the Intrade market’s performance implies about the usefulness of his Policy Analysis Market idea.

Note that I was contrasting the InTrade-Bin-Laden failure with the high expectations set by Robin Hanson, Justin Wolfers and James Surowiecki.

Also, other than statisticians, most people don’t have a probabilistic approach of InTrade’s predictions. That’s the big misunderstanding, which is one part of the big fail of the prediction markets.

It needs a way of summarizing people’s opinions into one collective verdict.

Sitting around with colleagues or e-mailing associates does not qualify. You need a specific mechanism to turn private judgments into a collective decision (e.g., a voting mechanism which permits abstention, a market mechanism, etc.).

ADDENDUM:

Not to put too much faith into the unscientific Gartner hype cycle, but it does feel like a time of backlash responding to some early over optimism. As cooler heads prevail, and the technology speaks for itself, I think we’ll start to see some solid success stories.

The industry failed because its aficionados (Robin Hanson, Justin Wolfers and James Surowiecki) oversold the benefits of the prediction markets. Overselling didn’t work in the past, and won’t work in the future. It’s time for Dave to wake up.

I like an analogy of Playstation versus Wii. Just like Wii opened up video gaming to a mass audience including women, I think there’s room for simpler/friendlier sites like smarkets that can open up gambling to a much wider audience than sites like betfair that seem to cater mostly to men familiar with gambling jargon.

4 more important factors:

1. Robin Hanson, James Surowiecki and Justin Wolfers have exaggerated the usefulness of the prediction markets, hence putting the bar too high.
2. Robin Hanson, James Surowiecki and Justin Wolfers were clearly the wrong leaders. They pushed the enterprise prediction markets, which proved to be a mirage. John Delaney was also the wrong leader to follow. TradeSports-InTrade was a big operator; InTrade is a niche player. All the leaders failed. All of them. Including David Pennock (who has the backing of an Internet giant). On top of that, the prediction market people let the San Francisco clown pollute our turf; he has now decamped for a greener turf, of course. We need better leaders. We need leaders with a better compass.
3. Real-money prediction markets are still illegal in most countries (including in David Pennock’s one).
4. North-American aficionados of the prediction markets (and that includes Paul Hewitt) have always lambasted UK-based BetFair, for silly reasons (just look up Pennock’s prose, above, for one instance), instead of making it the hero of our field. On the other side of the Atlantic, the BetFair management has never joined and supported the prediction market community.

The result is the Waterloo that David Pennock describes in his post, and no solution is proposed (other than the tired mantra, “yes, we can”). [Not a slam to research scientist Dave, just a parting way to get the debate going. ]

In the Fukushima case, Intrade was worse than a random generator. Outside the TepCo operators, nobody knew anything.

#fail

In my view, InTrade’s contracts on “current events” are over-cited in the economics literature and the business media.

[*] All of the people listed above are InTrade’s de-facto publicists.

Previously: InTrade CEO John Delaney apologizes for the ‘Mubarak Departure’ prediction market scandal. — {Mubarak Market Unwound} — [VICTORY]

Previously: InTrade’s Brand-New Rules for Departure Markets — [MARKET CONTRACT STATEMENTS]