Tag Archives: accuracy

Velocity + Inaccuracy

Posted on by
9

No Gravatar

One bit of criticism about my pamphlet (The Truth About Prediction Markets) goes like this: Velocity without accuracy is dumb.

That is not true.

Let&#8217-s imagine, for the sake of the exercise, that Barack Obama does not pick up Kathleen Sebelius to head HHS. The velocity argument remains valid: Fed by the vertical media (in this case, Yahoo News republishing the Associated Press), the prediction markets integrated expectations (informed by facts and expertise) much faster than the mass media did.

Any argument about the velocity of the prediction markets cannot be contradicted. No way.

Prediction markets didnt revolutionize decision-making -and will never do. However, they are a nice condiment to the classic forecasting toolkit.

Posted on by
Reply

No Gravatar

I have spent several hours re-reading the 2004 AEI-Brookings book, &#8220-Information Markets&#8221- (by which they mean &#8220-prediction markets&#8221-). It is a collection of un-enlightening research articles &#8212-except for the IEM article, which is outstanding, both on the factual and theoretical sides.

In the conclusion of their introduction, Robert Hahn and Paul Tetlock wrote that they want their readers to contemplate the idea that prediction markets could make a &#8220-big&#8221- difference and &#8220-revolutionize public- and private-sector decision-making&#8221-. Well, 4 years later, it is clear that those big dreams didn&#8217-t pan out. Not a single mass media outlet has praised the public prediction markets for their work on the 2008 US presidential election (I am taking about a post-mortem analysis about Election Day, not the primaries). Not a single one. (Not even Justin Wolfers.) And the number of corporations using enterprise prediction markets is still minute. The thinkers who wrote this book (&#8220-Information Markets&#8221-) all made the mistake to put the emphasis on accuracy instead of efficiency. That was the foundation flaw. We should reset and reboot the field of prediction markets.

Previously: The truth about prediction markets

Velocity is such a potent argument. Why dont we use it more, for Christs sake?

Posted on by
Reply

No Gravatar

I am re-reading a 2007 scientific article from Region Focus’ Vanessa Sumo:

– Ask The Market – Companies are leading the way in the use of prediction markets. The public sector may soon follow. – (PDF)

Here is what I see on the frontpage:

– &#8220-one or two weeks in advance&#8220-

– &#8220-even up to five weeks in advance&#8220-

Marketing-wise, velocity is a much more potent argument than the argument on accuracy. Who cares about an added accuracy of +2.7% (and that&#8217-s debated)? If any, that&#8217-s peanuts.

You cannot make a case against velocity. Impossible.

–

UPDATE: Put the PDF link in the address box of your browser (as opposed to clicking on it, or right-clicking on it).

http://www.richmondfed.org/publications/research/region_focus/2007/spring/pdf/feature1.pdf

The truth about prediction markets

Posted on by
Reply

No Gravatar

Come to the wonderful world of collective intelligence, wisdom of crowds, and prediction markets!&#8230- The sun shines bright, the market-generated predictions are vastly superior to the polls as election predictors, and the track record of the public prediction markets stretches as far as the eye can see. There are opportunities aplenty in the field of prediction markets, and the trading technology is cheap. Every working enterprise can have its own internal prediction exchange, and inside every exchange, a set of enterprise prediction markets that correctly predicts the future of business, which their happy, all-American CEO listens to. Life is good in the magic world of prediction markets&#8230- it&#8217-s paradise on Earth.

Ha! ha! ha! ha!&#8230- That&#8217-s what they tell you, anyway&#8230- &#8212-because they are selling an image (just as Bernie Madoff did). They are selling it thru their vendor websites, vendor conferences, vendor-inspired articles in blogs, newspapers and magazines, and interviews of vendor data-fed professors in the media.

The prediction market technology is not a disruptive technology, and the social utility of the prediction markets is marginal. Number one, the aggregated information has value only for the totally uninformed people (a group that comprises those who overly obsess with prediction markets and have a narrow cultural universe). Number two, the added accuracy (if any) is minute, and, anyway, doesn&#8217-t fill up the gap between expectations and omniscience (which is how people judge forecasters). In our view, the social utility of the prediction markets lays in efficiency, not in accuracy. In complicated situations, the prediction markets integrate expectations (informed by facts and expertise) much faster than the mass media do. Their accuracy/efficiency is their uniqueness. It is their velocity that we should put to work.

–

Here&#8217-s now our definition of prediction markets:

–

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative represents the imputed perceived likelihood of the partially uncertain future outcome (i.e., its aggregated expected probability). A 60% probability means that, in a series of events each with a 60% probability, the favored outcome is expected to occur 60 times out of 100, and the unfavored outcome is expected to occur 40 times out of 100.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism &#8212-with or without an automated market maker.

Prediction markets enable us to attain collective intelligence. Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that the traders bring when they agree on prices. The event derivative traders are informed by the primary indicators (i.e., the primary sources of information), like the polls, for instance. These informed speculators then execute their transactions based on their anticipations about the future &#8212-anticipations that will be either confirmed or infirmed.

The value of a set of prediction markets consists in the added accuracy that these prediction markets provide relative to the other meta predictive mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining these prediction markets, relative to the cost of the other meta predictive mechanisms. A highly accurate set of prediction markets has little value if some other meta predictive mechanism(s) can provide similar accuracy at a lower cost, or if very few substantial decisions are influenced by accurate predictions on its topic.

–

PS: I am updating a bit the content of this webpage, over time &#8212-so as to finesse the message.

Still, as noted, it was a good election for [the] prediction markets and another piece of evidence of their superiority over the pundit[s] (and at least parity with the poll).

Posted on by
Reply

No Gravatar

Dixit Nigel Eccles in a comment.

–

at least parity with the poll

–

I agree with the above.

–

their superiority over the pundits

–

What documented evidence do you have about that, mister the cocky entrepreneurial Scotsman?

John Tierney linked to that Huffington Post that listed the pundits&#8217- predictions about the total number of electoral votes that each presidential candidate would take. But I disagree with that way of predicting the electoral college and assessing these predictions. With this completely flawed method, if you are damn wrong on a state and damn wrong (in the opposite way) about another state that has the exact same number of electoral votes, then you are a bright genius worth the Nobel prize of forecasting. Gimme a break. Enough with that voodoo way of assessing predictions about the electoral college. Do the assessment state by state.

InTrade and HubDub got lucky that their 2 mistakes (so to speak, in a non-probabilistic way) on Missouri and Indiana (both with 11 electoral votes) canceled themselves perfectly. IT WAS PURE LUCK. If their 2 mistakes had been made in the same direction (say, betting on Obama with the outcome going eventually to McCain), and/or their 2 mistakes had been done on 2 very dissimilar states (say, one with 6 electoral votes and the other one with 27 electoral votes), then we would have had reporters and bloggers bashing the prediction markets for the whole month of November.

–

Dont pump up the features of the prediction markets -instead, put the emphasis on their benefits.

Posted on by
6

No Gravatar

John Tierney and Jed Christiansen are making the same mistake: they think that people and experts should be impressed by the information aggregation functionality of the prediction markets. They are not &#8212-people still prefer reading Nate Silver and Electoral-Vote.com over InTrade, and the political experts have not added InTrade in their toolbox. (On this last point, do read the very last sentence of that interview.)

You won&#8217-t be impacting if you publish enthusiastically about the features of the prediction markets &#8212-yes, they do incorporate the latest news quickly, they quantify reasonable anticipations, they output probabilities, and they are relatively unbiased. You will be impacting the day you are able to demonstrate the benefits of the prediction markets &#8212-for people, on one hand, and for the experts, on the other hand.

This would require a new focus, and a much bigger effort.

The social utility of most prediction markets is minimal &#8212-busy people (who don&#8217-t have time to read extensively the news) get relatively objective probabilities, real quick. But very few companies are using enterprise prediction markets, as of today. If these new IAM tools were magical (as some sur-excited free-market proponents think they are), all the Fortune-500 companies without any exception of any kind would be using them today.

If you want to discover the true benefits of the prediction markets, you should first be able to rank them by degree of utility. Which ones are more useful than others? Why? To answer this last question, you have to lay out the panorama of all the information sources that people and expert have access to, these days. What were the specific instances where the prediction markets were a tie breaker between the experts and the mass media, or between the decision makers and the experts, or between 2 opposite groups of experts? You should build an airtight, documented case. I haven&#8217-t seen such a case, yet. If some of my readers are interested in such a project, let&#8217-s talk.

–

The Intrade bettors expected Mr. Obama to end up with 364 votes in the Electoral College -one less than he actually got.

Posted on by
5

No Gravatar

My remark to John Tierney:

InTrade got it [almost] spot on because they were wrong on Missouri (which was predicted to go for Obama but went to McCain) and wrong too on Indiana (which was predicted to go for McCain but went to Obama) —and those 2 opposite mistakes canceled themselves because those 2 states have the exact same number of electoral votes (11). Hence, I disagree with your method.

–

–

APPENDIX:

Here&#8217-s a visual post-mortem of the 2008 US presidential elections.

Pay attention to Missouri and Indiana.

–

A) InTrade, on November 5, 2008 (screen shot taken at 2:00 am):

–

–

Prediction Markets &amp- State Polls, on November 4, 2008:

–

–

B1) Prediction Markets (on November 4, 2008)

–

–

InTrade (screen shot taken at mid-day ET, November 4, 2008):

–

InTrade (screen shot taken in the morning, November 4, 2008):

–

BetFair (screen shot taken in the morning, November 4, 2008):

–

HubDub (screen shot taken in the morning, November 4, 2008):

–

–

–

–

–

B2) State Polls (on November 4, 2008)

–

–

Karl Rove (on November 4, 2008):

–

CNN (on November 4, 2008):

–

Pollster (on November 4, 2008):

–

Electoral-Vote.com (on November 4, 2008):

–

Nate Silver (on November 4, 2008):

–

–

PREDICTION MARKET PROBABILITIES

–

Explainer On Prediction Markets

–

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative represents the imputed perceived likelihood of the partially uncertain event (i.e., its aggregated expected probability). A 60% probability means that, in a series of events each with a 60% probability, the favored outcome is expected to occur 60 times out of 100, and the unfavored outcome is expected to occur 40 times out of 100.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism &#8212-with or without an automated market maker.

Prediction markets enable us to attain collective intelligence. Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that the traders bring when they agree on prices. The event derivative traders are informed by the primary indicators (i.e., the primary sources of information), like the polls, for instance. These informed speculators then execute their transactions based on their anticipations about the future &#8212-anticipations that will be either confirmed or infirmed.

The value of a set of prediction markets consists in the added accuracy that these prediction markets provide relative to the other forecasting mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining these prediction markets, relative to the cost of the other forecasting mechanisms. According to Robin Hanson, a highly accurate prediction market has little value if some other forecasting mechanism(s) can provide similar accuracy at a lower cost, or if very few substantial decisions are influenced by accurate forecasts on its topic.

–

More Info:

– The Best Resources On Prediction Markets = The Best External Web Links + The Best Midas Oracle Posts

– Prediction Market Science

– The Midas Oracle Explainers On Prediction Markets

– All The Midas Oracle Explainers On Prediction Markets

–

My open challenge to AskMarkets co-founder George Tziralis

Posted on by
2

No Gravatar

Dear George,

Congrats for the launch of AskMarkets. Best wishes to your prediction exchange and consulting firm.

Here&#8217-s the perfect opportunity to ask you the &#8220-question that kills&#8221-:

What was the social utility of the political election prediction markets during the 2008 campaign?

In other words, why should the media have informed people about the InTrade probabilities at a time Nate Silver did a near-perfect job forecasting the 2008 US elections?

What&#8217-s the added value of the political election prediction markets over the poll aggregators?

Can you cite one prediction market (other than the &#8220-who&#8217-s gonna become president?&#8221- prediction market) that has a high social utility?

Each time I ask this question to one of the prediction market luminaries (or so they think they are), I get back the same glance I would get from a dead trout &#8212-so I would appreciate if you could attempt to answer my question by publishing a blog post on Midas Oracle.

Best regards,

Chris Masse, bombastic blogger

http://www.midasoracle.org/