-

New Hampshire – The Democrats

-

The Hillary Clinton event derivative was expired to 100.

-

New Hampshire – The Republicans

-

The John McCain event derivative was expired to 100.

-

“a handful of methodological missteps and miscalculations combined to undermine the accuracy of predictions about presidential primary winners in New Hampshire and three other states.”

Via Mister the Great Research Scientist David Pennock –who is an indispensable element of the field of prediction markets.

As I blogged many times, prediction markets react to polls… See the addendum below… – [UPDATE: See also Jed's comment.] – Prediction markets should not be hyped as crystal balls, but simply as an objective and continuous way to aggregate expectations. So, if you think of it, their social utility is much smaller than what the advocates of the “idea futures”, “wisdom of crowds” or “collective intelligence” concepts told us. Much, much, much, much smaller… They all make the mistake to put accuracy forward. (By the way, somewhat related to that issue, please go reading the dialog between Robin Hanson and Emile Servan-Schreiber.)

-

Addendum

-

California Institute of Technology economist Charles Plott:

What you’re doing is collecting bits and pieces of information and aggregating it so we can watch it and understand what people know. People picked this up and called it the “wisdom of crowds” and other things, but a lot of that is just hype.

-

New Hampshire – The Democrats

-

The Hillary Clinton event derivative was expired to 100.

-

New Hampshire – The Republicans

-

The John McCain event derivative was expired to 100.

-

-

Our good friend Barry Ritholtz.has persuaded himself that our real-money prediction markets suffer from an irremediable and fatal problem: liquidity on political event derivative markets is too thin for smart Wall Street people like him to take their market-generated probabilities seriously. Barry Ritholtz is keen to tout oranges–apples comparisons: the NYSE volume versus the Obama–Clinton volume at InTrade. It’s a bullshit argument, but he managed to persuade some gullible journalists writing for some clueless mainstream media that thin liquidity was responsible for the New Hampshire upset —and else.

Barry, if you had 1,000,000,000 trades on the New Hampshire prediction market, you’d still have an inaccurate prediction. The polls were wrong, and there’s nothing … NOTHING… that the InTrade and BetFair traders could have done to get this election right. Get over it, Barry. Traders are not magicians.

-

[For why the polls were wrong, see: The New York Times, Zogby, Rasmussen, Gallup...]

-

John McQuaid of Wired.

1. It’s incoherent to start a rant against prediction markets by this upbeat line, “Prediction markets can be spookily accurate.”
2. He blames the New Hampshire upset on poor liquidity. Where is the scientific evidence of that? Total invention by our good friend Barry Ritholtz. The New Hampshire prediction markets were wrong because the advanced, primary indicators (the polls) were wrong. As simple as that. [For why the polls were wrong, see: The New York Times, Zogby, Rasmussen, Gallup...]
3. Prediction markets “have a lot of political junkies but few real insiders or outsiders, so they’re not very good at catching something the polls might miss.” Hummm… Most of the “real insiders” don’t keep scoops for themselves (if and when they have some), they are too happy to act as a source for some thirsty journalists or bloggers, so as to have their name printed somewhere. Hence, the political junkies would be able to aggregate any kind of extraordinary information —if that were to happen.
4. How could the prediction markets “get out ahead of conventional wisdom”? It’s impossible, other than by reversing our psychological arrow of time (remembering the future, instead of the past). At the contrary, the job of the prediction markets is to quantify exactly that so-called “conventional wisdom”. They won’t go further, and we’re happy to run with that, because, that way, we are not prisoner of the bias of a handful of experts. Plus, prediction markets give us an objective probability of event outcome —a thing that individual experts can’t give us.

-

The excerpt below is good enough, though:

[...] But forecasting also needs more so-called noise traders, who do business with almost no information. Noise traders boost accuracy by increasing volume and the potential profits of informed traders. Diversity helps, too. If you can get different types of people to play, experts say, not only do you get a bigger pool and more information, but differing random guesses will cancel each other out, leaving real signals to rise above the noise. Plus, if you have a critical mass of investors with a variety of backgrounds, locations, and interests, they are less likely to move as a herd. [...]

-

Despite that one blip on the radar [New Hampshire], Strumpf said futures are still the best way to predict the way things will go from here.

Spot the SIDEBAR (which is not located on the sidebar, actually), and click on the little square, just below “video”, to watch the report.

-

All told, prediction market data tend to confirm that Mr. Giuliani’s recent decline is due to a poor campaign, rather than a poor strategy.

I have just read the 3-page New York Times story on Rudy Giuliani’s “dizzying free-fall” (who has not yet dropped out and endorsed John McCain, as I write this). And, as I understand the two NYT writers, they attribute his failure to both a strategic mistake and a poor campaign.

1. “many advisers and political observers point to the hubris and strategic miscalculations that plagued his campaign.” … “a fateful decision” … “skipping the first four or five caucuses and primaries” …
2. “he in fact competed hard in New Hampshire, to remarkably poor effect.“

As of today, I am not yet convinced that Justin Wolfers saw something in the prediction markets that the political analysts didn’t see in the polls and in the votes. Thus, I will find other Giuliani reports. I’ll see. What is needed is an independent judgment on this.

I’m happy to report progress in building the architecture and content of our group blog, Midas Oracle. Let me introduce you to our 2 master pages: “PREDICTIONS” and “BEST”.

—

—

#1. PREDICTIONS

http://www.midasoracle.org/predictions/

= our explainer on prediction markets + the best charts of prediction markets

Following the New Hampshire brouhaha, I believe that the way to go is to systematically associate (when time and space permit) a short probability explainer to any published chart of prediction market. You can see such an association at the top of our blog frontpage. I will do that in some blog posts, too, in the future.

Political charts from InTrade and NewsFutures are published there. Charts from more exchanges will be added there over the coming weeks, months, and years. Suggestions are welcomed.

Thanks to Michael Giberson and Emile Servan-Schreiber for some tiny input in establishing that brand-new page.

—

—

#2. BEST

http://www.midasoracle.org/best/

= the best external weblinks on prediction markets + the best Midas Oracle posts on prediction markets

This file has been quite popular —see the web stats report, in the appendix. I will, of course, augment this file with many more pointers, in the coming weeks, months, and years. Suggestions are welcomed.

—

—

#3. ARCHIVES

http://www.midasoracle.org/archives/

You can now search the archives of the Midas Oracle blog posts —by date, by category, by tag, by poster, etc. Bo Cowgill will now be able to dig the Midas Oracle archives to unearth all “the inaccuracies” that I have printed “over the years”. Good luck for your quest, Bo.

—

—

#4. LOOKING FORWARD

More pageviews for the blog.

More coverage of the prediction markets (as opposed to coverage of the prediction market industry) —on the other two blogs (Midas Oracle .NET and Midas Oracle .COM).

More academic paper reviews.

A sense of what could be prediction market journalism (pioneered by Justin Wolfers in the Wall Street Journal).

A discussion on the usefulness of a prediction market plugin for WordPress.

And, maybe, the building of an advisory board for Midas Oracle —if that makes sense and serves a purpose.

Step one towards our ultimate goal: WORLD DOMINATION.

—

—

APPENDIX

Below are pageviews of the most popular Midas Oracle webpages, since September 2006. For the posts (as opposed to the pages), to get a more complete picture, you should add the number 1,050 (estimation) to the numbers below —to account for the feed subscribers (900) and the blog frontpage readers (150), who both also consumed the posts in question.

Please note that these stats are from Google Analytics. When assessing the popularity of Midas Oracle against other websites, do use the Google Analytics numbers, and not numbers from other web stats services. (I also have server stats, whose numbers are much more impressive, but they don’t count the same things.) Thanks.

—

I am alerted today that the BetFair blog has updated its infamous Michigan story with a new compound chart bearing a clearer label. It reads now:

Republican nomination – The race so far

I’ll have some comments, below the chart, but first a technical note. The new chart posted is a 527-KB BMP image. I have replaced it with a 32-KB JPG image. The BetFair blog is not run professionally. Any web publisher knows that images should be reduced to the max. That’s the ABC of web publishing. (And to add insult to injury, I noted previously the technical bizarrery that the two professor Leighton Vaughan-Williams’s stories never appeared in the BetFair blog feed.)

For you information, I have updated all my previous blog posts on the topic with an addendum re-publishing this new chart.

—

UPDATED ANALYSIS OF THE BETFAIR BLOG FIASCO:

1. Professor Leighton Vaughan-Williams should have defined what he means by “betting markets”, in his story. In the past (see the addendum of that story), prof Leighton Vaughan-Williams used two types of cocktail —one including all betting markets (traditional bookmaker odds and exchange odds), one including only the BetFair odds. He should publish an addendum to his story defining exactly what he means by “betting markets”, this time.
2. The BetFair blog editor should not have pasted a BetFair compound chart behind the writer’s back. It’s a big no-no in editing. Again, another proof (in a long list) that the Betfair blog is not run professionally.
3. If a chart were to be inserted on top of LVW’s story (with his consent, we hope), it should have been the expired chart(s) of the Michigan primary, since that’s the heart of LVW’s story.
4. The fact that the BetFair blog editor pasted (behind LVW’s back) a BetFair chart lead the readers (like Niall Or’Connor and me) to conclude that professor Leighton Vaughan-Williams means “the BetFair betting markets” when he writes about “the betting markets”. This is probably not the case, but nobody knows for sure —see my point #1 for the need of an explainer on this.
5. Now, if professor Leighton Vaughan-Williams means “the BetFair betting market” (I assign a low probability on this scenario), then the story looks bad. The story is bullish on the fact that the Mitt Romney event derivative (for the Michigan primary) was predictive. The election-day chart that I published yesterday evening (and republished below) shows Mitt Romney being the favorite starting at 3:00 PM EST on election day… Kind of a stretch to claim victory for the BetFair betting markets. I’m still waiting for BetFair to send me the full, historical chart on the Michigan primary.
6. BetFair should publish all expired charts —just like InTrade-TradeSports are doing. See my new page, re-publishing some important expired prediction market charts. That way, any controversy could be settled more quickly.
7. With all due respect to him, it looks bad on professor Leighton Vaughan-Williams for giving his writings to a corporate blog where the publisher and editor’s names are not listed anywhere, and whose overall content quality is feeble —to say the least. Especially since we read the testimony of a furious Betair blog writer, who described the BetFair blog editor as anonymous, incompetent and tyrannical.
8. Besides Niall O’Connor’s critical comments, professor Leighton Vaughan-Williams’s story on the BetFair blog has attracted a negative comment, calling his argument “questionable to say the least“, and asking (as I am doing on this current post) for more data to be published in an addendum.
9. It looks bad on the BetFair management for publishing completely crappy stories like that. It damages the BetFair brand. I should tell my readers, though, that the BetFair-TradeFair managers (like Michel Robb, Tony Clare, Mark Davies, David Jack, Robin Marks, etc.) are highly professional, efficient, law-abiding, forward-looking, helpful, ethical, polite, and respectful. It is a real pity that the BetFair blog tarnishes BetFair’s reputation.
10. Betair should focus on being a prediction market resource for journalists and bloggers. As of today, they still don’t provide on their website dynamic charts and expired charts.
11. As I repeated many times on Midas Oracle, prediction market journalism is hard, complex and costly. It can’t be done by any living organism (hermaphrodite or not ) simply equipped with a computer and an Internet connection.

—

As an addendum, I re-publish here the election-day Michigan chart (on the Republican side). As I said, I’m still waiting for BetFair to send me the full, historical chart. You can see, on this Republican-side chart, Mitt Romney (in red) as the Comeback Kid —starting at 3:00PM EST on election day (that’s 8:00 PM, British time, on the chart).

For your information, here’s what professor Leighton Vaughan-Williams wrote. As I said, an explainer from him is needed to determine whether he means the “betting markets” in general (with, or without, BetFair included?) or the “BetFair betting markets”.

Professor Leighton Vaughan-Williams on the official BetFair blog:

[...] Those taking the same advice on Tuesday evening [2008-01-15 = date of the Michigan primary] were similarly well rewarded as well-backed Mitt Romney stormed into clear favouritism in the markets and a comfortable victory at the polls. After a blip in the New Hampshire Democratic primary the old certainties – that election favourites tend to win elections – was re-established.

As in the Republican New Hampshire primary, the polls and pundits had declared the race between Senator McCain and Governor Romney as a toss-up while the betting markets pointed to a comfortable victory in both cases for the eventual winners. Once again, in the battle of the polls, pundits and markets, the power of the betting markets to assimilate the collective knowledge and wisdom of the crowd had prevailed. [...]

As for the InTrade “betting markets”, if that’s what professor Leighton Vaughan-Williams means (solely, or among others), they show a strong support for Mitt Romney in the last 2 days (which includes election day). Kind of a stretch to claim a victory for the “betting markets”. Also, it would be funny to have the (anynomized) InTrade data interpreted on the blog of another exchange (BetFair, a competitor of InTrade-TradeSports) to hint about the alleged strength and accuracy of the BetFair “betting markets”. That would be the last drop that breaks the water bucket. Another reason why professor Leighton Vaughan-Williams should come forward to explain what he means by “betting markets” in his story. Does he mean the “InTrade betting markets”???

(FYI, the Mitt Romney event derivative was expired to 100.)

Psstt… Sounds like a vertical line is lacking on this chart… Look at the right end… Bizarre.

—

NEXT: No more anonymized trading data, please. State your source(s).

—

[...] Notice that when we have a surprise victory in a primary, like Clinton in New Hampshire, much of the talk revolves on why the pundits, polls and prediction markets all “failed.” Meanwhile in sports when we see a surprise victory, like the New York Giants over Dallas and then again in Green Bay, the focus is on what the Giants did right and the Cowboys and Packers did wrong. Sports fans understand probabilities much better than political junkies—upsets happen occasionally, just as they should.

Previously: Defining Probability in Prediction Markets – by Panos Ipeirotis – 2008

[...] Interestingly enough, such failed predictions are absolutely necessary if we want to take the concept of prediction markets seriously. If the frontrunner in a prediction market was always the winner, then the markets would have been a seriously flawed mechanism. [...]

Previously: Can prediction markets be right too often? – by David Pennock – 2006

[...] But this begs another question: didn’t TradeSports call too many states correctly? [...] The bottom line is we need more data across many elections to truly test TradeSports’s accuracy and calibration. [...] The truth is, I probably just got lucky, and it’s nearly impossible to say whether TradeSports underestimated or overestimated much of anything based on a single election. Such is part of the difficulty of evaluating probabilistic forecasts. [...]

Previously: Evaluating probabilistic predictions – by David Pennock – 2006

[...] Their critiques reflect a clear misunderstanding of the nature of probabilistic predictions, as many others have pointed out. Their misunderstanding is perhaps not so surprising. Evaluating probabilistic predictions is a subtle and complex endeavor, and in fact there is no absolute right way to do it. This fact may pose a barrier for the average person to understand and trust (probabilistic) prediction market forecasts. [...] In other words, for a predictor to be considered good it must pass the calibration test, but at the same time some very poor or useless predictors may also pass the calibration test. Often a stronger test is needed to truly evaluate the accuracy of probabilistic predictions. [...]

The current trend to promote almost illiquid betting markets as being predictive, looks set to backfire bigstyle. In a nutshell, to date the so called “prediction markets” have called it wrong in New Hampshire (Democrats), Michigan (Republicans) and South Carolina (Republicans). Prediction market advocates, in many instances weighed down with the baggage of being too closely associated with the prediction market industry, are guilty of churning out subjective, biased research, in an attempt to promote the industry and lure traders into the markets. Unscientific comparisons are continually made between prediction markets and polls, and, moreover, markets are deemed to have been predictive, when all they have actually done is [respond] to the unwinding of the vote count. The publication of such utter garbage does nothing but a big disservice to the fledgling prediction market industry. Moreover, it is unfortunate that prediction market industry watchers, who purport to be independent and subjective, indulge the writers of such nonsense, in the process revealing themselves to be nothing more than placemen and sycophants of unquestioning loyalty. [...]

Names, please, Niall.

1. Who are those corrupted “prediction market advocates”?
2. Who are those rotten “prediction market industry watchers”?

By the way… I will have the charts of the expired BetFair contracts, soon, I hope; and I will publish them here for all to see.