Assessing Probabilistic Predictions 101

No Gravatar

Lance Fortnow:

[…] Notice that when we have a surprise victory in a primary, like Clinton in New Hampshire, much of the talk revolves on why the pundits, polls and prediction markets all &#8220-failed.&#8221- Meanwhile in sports when we see a surprise victory, like the New York Giants over Dallas and then again in Green Bay, the focus is on what the Giants did right and the Cowboys and Packers did wrong. Sports fans understand probabilities much better than political junkies—upsets happen occasionally, just as they should.

Previously: Defining Probability in Prediction Markets – by Panos Ipeirotis – 2008

[…] Interestingly enough, such failed predictions are absolutely necessary if we want to take the concept of prediction markets seriously. If the frontrunner in a prediction market was always the winner, then the markets would have been a seriously flawed mechanism. […]

Previously: Can prediction markets be right too often? – by David Pennock – 2006

[…] But this begs another question: didn’t TradeSports call too many states correctly? […] The bottom line is we need more data across many elections to truly test TradeSports’s accuracy and calibration. […] The truth is, I probably just got lucky, and it’s nearly impossible to say whether TradeSports underestimated or overestimated much of anything based on a single election. Such is part of the difficulty of evaluating probabilistic forecasts. […]

Previously: Evaluating probabilistic predictions – by David Pennock – 2006

[…] Their critiques reflect a clear misunderstanding of the nature of probabilistic predictions, as many others have pointed out. Their misunderstanding is perhaps not so surprising. Evaluating probabilistic predictions is a subtle and complex endeavor, and in fact there is no absolute right way to do it. This fact may pose a barrier for the average person to understand and trust (probabilistic) prediction market forecasts. […] In other words, for a predictor to be considered good it must pass the calibration test, but at the same time some very poor or useless predictors may also pass the calibration test. Often a stronger test is needed to truly evaluate the accuracy of probabilistic predictions. […]

The Michigan primary as seen thru the BetFair prediction markets

No Gravatar

Thanks to Michael Robb from BetFair, I can show you the charts of the last day of trading on Michigan&#8217-s election day. You can see, on the Republican chart, Mitt Romney (in red) as the Comeback Kid &#8212-starting at 3:00PM EST (that&#8217-s 8:00 PM, British time, on the chart).

Rep Michigan BetFair

Dem Michigan BetFair

Here is the full, historical, compound chart that I already published:

Michigan BeFair

Is that the Michigan primary, really? Bizarre.There is something I don&#8217-t get. Compare with the InTrade charts.

Fat fingers at BetFair? Did they mess with the legend? Or is this compound chart about another primary? (Which one, then??) Bizarre, bizarre.

&#8212-

TAKEAWAY ABOUT THE BETFAIR BLOG FIASCO:

  1. It looks bad for the BetFair blog editor, who pasted that compound chart on top of professor Leighton Vaughan-Williams&#8217-s story. One week later, it is still impossible for me to comprehend what that compound chart is supposed to represent, and what it is doing on that page. Big mystery.
  2. It is a shame that the BetFair blog hasn&#8217-t published an update note stating that that compound chart was pasted there by the editor (behind prof LVW&#8217-s back) and explaining what the writer meant by &#8220-betting markets&#8221-. Still a puzzle.
  3. With all due respect to him, it looks bad on professor Leighton Vaughan-Williams for giving his writings to a corporate blog where the publisher and editor&#8217-s names are not listed anywhere, and whose overall content quality is feeble &#8212-to say the least.
  4. And, by the way, how come professor Leighton Vaughan-Williams&#8217-s posts are always excluded from the general BetFair blog feed? Is that a technical glitch? Or is it by design? Bizarre.
  5. The BetFair blog is not a serious publication. Period. I wonder whether I should link again to it, in the future. Probably not.

&#8212-

UPDATE: The BetFair blog has added a new label on the infamous compound chart&#8230-

Compound chart - BetFair blog fiasco

&#8212-

NEXT: Did the BetFair blog use trading data from InTrade to hint at BetFair&#8217-s accuracy??

The Formula for Internet Site Success

Jakob Nielsen:

The formula for website success is:

  • B = V ? C ? L

Where:

  • B = amount of business done by the site-
  • V = unique visitors coming to the site-
  • C = conversion rate (the percentage of visitors who become customers)- (Note that the concept of conversion applies not only to e-commerce sites, but to any site where there is something you want users to do.)
  • L = loyalty rate (the degree to which customers return to conduct repeat business).

Read the previous blog posts by Chris. F. Masse:

  • Hilarious.
  • Prediction market sessions of the O’Reilly Money-Tech Conference suffer fatally from the absence of the world’s most knowledgeable, most innovative and most trustworthy prediction market expert.
  • Bet2Give Presidential Widget —REDUX
  • Professor Leighton Vaughan-Williams becomes embroilled in BetFair blog controversy.
  • Bet2Give Presidential Widget
  • Did the BetFair blog use trading data from InTrade to hint at BetFair’s accuracy??
  • WordPress powers the MSM’s blogs: NY Times, WSJ, CNN, Fox, Time and People.

Democratic and Republican caucuses in Nevada + Republican primary in South Carolina

Here are the event derivative charts of expired InTrade contracts.

[Psstt&#8230- I have high hopes about being able to publish the charts of expired BetFair contracts, too, soon. :-D ]

&#8212-

&#8212-

Democratic and Republican caucuses in Nevada

&#8212-

Democratic caucus in Nevada (the Hillary Clinton event derivative was expired at 100):

Dem Nevada Clinton

Dem Nevada Obama

Dem Nevada Edwards

&#8212-

Republican caucus in Nevada (the Mitt Romney event derivative was expired at 100):

Rep Nevada Romney

Rep Nevada McCain

&#8212-

&#8212-

Republican primary in South Carolina

&#8212-

Republican primary in South Carolina (the John McCain event derivative was expired to 100):

Rep SC McCain

&#8220-Field&#8221- = Mike Husckabee

Rep SC Field

Rep SC Romney

Rep SC Thompson-F

Source: InTrade

Read the previous blog posts by Chris. F. Masse:

  • Super Bowl XLII
  • The Glorious Incertitude Of Sports
  • TradeSports Cost Of Transaction
  • The prediction market industry needs people who have balls.
  • Is it good to have a prediction market melting pot of academics and businesses?
  • Is it time to buy some Michael Bloomberg event derivatives?
  • Professor Leighton Vaughan-Williams excluded from the BetFair blog feed

NIALL OCONNOR ATTACKS THE INTELLECTUAL HONESTY OF THE PREDICTION MARKET RESEARCHERS.

No Gravatar

Niall O&#8217-Connor (who maybe ate grilled snake for his Sunday morning breakfast :-D ):

The current trend to promote almost illiquid betting markets as being predictive, looks set to backfire bigstyle. In a nutshell, to date the so called &#8220-prediction markets&#8221- have called it wrong in New Hampshire (Democrats), Michigan (Republicans) and South Carolina (Republicans). Prediction market advocates, in many instances weighed down with the baggage of being too closely associated with the prediction market industry, are guilty of churning out subjective, biased research, in an attempt to promote the industry and lure traders into the markets. Unscientific comparisons are continually made between prediction markets and polls, and, moreover, markets are deemed to have been predictive, when all they have actually done is [respond] to the unwinding of the vote count. The publication of such utter garbage does nothing but a big disservice to the fledgling prediction market industry. Moreover, it is unfortunate that prediction market industry watchers, who purport to be independent and subjective, indulge the writers of such nonsense, in the process revealing themselves to be nothing more than placemen and sycophants of unquestioning loyalty. […]

Names, please, Niall.

  1. Who are those corrupted &#8220-prediction market advocates&#8221-?
  2. Who are those rotten &#8220-prediction market industry watchers&#8221-?

By the way&#8230- I will have the charts of the expired BetFair contracts, soon, I hope- and I will publish them here for all to see.

Midas Oracle .ORG » Blog Archive » The prediction market industry needs people who have balls.

Squirrel

Previously: Squirrel attack prediction market, anyone??

Read the previous blog posts by Chris. F. Masse:

  • Super Bowl XLII
  • The Glorious Incertitude Of Sports
  • TradeSports Cost Of Transaction
  • Is it good to have a prediction market melting pot of academics and businesses?
  • Is it time to buy some Michael Bloomberg event derivatives?
  • Professor Leighton Vaughan-Williams excluded from the BetFair blog feed
  • NIALL O’CONNOR ATTACKS THE INTELLECTUAL HONESTY OF THE PREDICTION MARKET RESEARCHERS.

An Atlanta hair saloon is one of the many satisfied clients of WeatherBill.

Via Steve Roman, The Atlanta Journal – Constitution:

[&#8230-] Atlanta hair salon owners Ray Luciano and Spencer Malay recently reeled in a bundle — tens of thousands of dollars — after taking out insurance-like coverage called &#8220-weather derivatives.&#8221- Their Buckhead salon does well when it rains, drawing walk-ins and overflow from the nearby mall and movie theater. &#8220-If it&#8217-s a sunny day, they [shoppers] tend to do outside stuff, not go to the movies, not go shopping, not get their hair done,&#8221- Luciano said. [&#8230-] In the last nine months, the owners at Spencer Malay Hair have taken out four contracts, and Lady Luck has smiled on them three times. The most recent was for the holiday season between mid-November and just before Christmas. &#8220-We put a dry-season contract on it because we hired a couple of new stylists,&#8221- Luciano said. The proceeds, which he declined to talk about in detail, allowed them to &#8220-pay our staff while business was not as good as good as it could have been.&#8221- [&#8230-]

Previously: Eric Zitzewitz on WeatherBill

Read the previous blog posts by Chris. F. Masse:

  • Davos – World Economic Forum
  • CME Group = Chicago Mercantile Exchange + Chicago Board Of Trade
  • Democratic and Republican caucuses in Nevada + Republican primary in South Carolina
  • The BetFair blog is not a serious publication.
  • MICHIGAN PRIMARY @ BETFAIR: Niall O’Connor asks the very pertinent question.
  • One thing John Delaney and his Irish employees at InTrade-TradeSports can learn from the BetFair-TradeFair folks at HammerSmith.
  • BetFair compound chart on the Michigan primary

The BetFair blog is not a serious publication.

No Gravatar

I just got it that professor Leighton Vaughan-Williams&#8217-s story published on the official BetFair blog and the BetFair compound chart published on top of his story should be understood independently from each other, as the BetFair chart was pasted there by the BetFair blog editor.

(I have updated all my previous posts on the topic with the explainer above.)

&#8212-

TAKEAWAY:

  1. Professor Leighton Vaughan-Williams talked about &#8220-betting markets&#8221- in his story. Those &#8220-betting markets&#8221- were not defined precisely on that page.
  2. The BetFair compound chart, pasted by the BetFair blog editor on top of Professor Leighton Vaughan-Williams&#8217- story, has lead readers (like the ultra-vigilant Niall O&#8217-Connor) to believe that &#8220-betting markets&#8221- meant the BetFair betting markets.
  3. It was probably not the case, in the writer&#8217-s mind. To be sure, though, an explainer should be issued by the BetFair blog.
  4. The BetFair blog editor has an omelette on his/her face.
  5. I told you many times that the Betfair blog is a piece of crap.
  6. A writer for this blog told us so, too.
  7. As I said many times, web journalism is not for anybody. It&#8217-s a specialty trade. It requires vertical skills. And, more precisely, prediction market journalism, in our digital times, is difficult and complex. It&#8217-s sad to see that the BetFair-TradeFair executives haven&#8217-t computed that.

&#8212-

UPDATE: The BetFair blog has added a new label on the infamous compound chart&#8230-

Compound chart - BetFair blog fiasco

&#8212-

NEXT: Did the BetFair blog use trading data from InTrade to hint at BetFair&#8217-s accuracy??

One thing John Delaney and his Irish employees at InTrade-TradeSports can learn from the BetFair-TradeFair folks at HammerSmith.

POLITENESS.

Tomorrow’s overnight cards

Betfair Customer Services 18 Jan 19:14
Please be aware that all our overnight cards for Saturday’s racing are currently suspended while we investigate a potential technical issue. Please accept our apologies for the inconvenience this will cause. At this stage it is unclear when they will be re-opened.

Betfair Customer Services 18 Jan 22:54
All tomorrow&#8217-s cards are now re-opened. Once again, please accept our apologies for any inconvenience caused.

Bombastic blogger Chris Masse could take some lessons from them, too. :-D

Read the previous blog posts by Chris. F. Masse:

  • Davos – World Economic Forum
  • CME Group = Chicago Mercantile Exchange + Chicago Board Of Trade
  • Democratic and Republican caucuses in Nevada + Republican primary in South Carolina
  • The BetFair blog is not a serious publication.
  • MICHIGAN PRIMARY @ BETFAIR: Niall O’Connor asks the very pertinent question.
  • BetFair compound chart on the Michigan primary
  • New Hampshire fiasco blamed on lack of InTrade traders’ diversity