Tag Archives: Forecasting

Obstacles to Prediction Market Adoption

Posted on by
Reply

No Gravatar

BusinessWeek:

Harrah&#8217-s is setting up a pilot prediction market to forecast customer activity in one of its domestic casino operations. […]

Since the power of prediction markets hinges on effectively tapping into cognitive diversity throughout an organization, Page also argues convincingly that if members of a group do not have enough diversity in their perspectives, prediction markets can actually produce dismal results. […]

Until now, few of the companies sponsoring successful pilots or tests have deployed prediction markets on a broad or sustained basis. Why not? One explanation is that prediction markets are deeply subversive. After all, lots of midlevel executives are consumed with the task of forecasting. If prediction markets do a better job of it, doesn&#8217-t that discredit the efforts (and perhaps even the motives) of these executives? But as prediction markets shift their focus toward new knowledge creation, they may become less threatening within corporations. […]

–

I don&#8217-t buy this explanation &#8212-nor do I buy that other one.

My view is that we haven&#8217-t yet demonstrated clearly when and how prediction markets can be useful.

–

MBAs on Enterprise Prediction Markets

Posted on by
Reply

No Gravatar

Alan H.:

During the class, Adam Siegel, the founder of Inkling Markets, a prediction markets consulting firm, spoke about his experiences. Of the benefits, he said that prediction markets bring clarity around information, prevent political fudging and backstabbing regarding information. Nobody is the whistleblower for challenging optimistic assumptions, rather, &#8220-it&#8217-s the market.&#8221- […]

Given that many executives and managers want to hide their poor performance, I asked Adam about who typically approaches his firm. He responded that he is usually approached by either third parties who have no P&amp-L responsibility, such as strategic planning groups, or forward thinking managers who are sick of bad forecasts being submitted. […]

–

Read Adam Siegel&#8217-s post about his intervention.

–

Prediction markets feed on facts and expertise.

Posted on by
1

No Gravatar

Via Yahoo! research scientist David Pennock of Odd Head and YooPick, the dear honorable Duncan Watts:

In part because of disappointing findings such as this, an increasingly popular substitute for expert opinions are so-called &#8220-prediction markets,&#8221- in which individuals buy and sell contracts on various outcomes, such as football game point spreads or presidential elections. The market prices for these contracts then effectively aggregate the knowledge and judgment of the many into a single prediction, which often turns out to be more accurate than all but the best individual guesses.

But even if these markets do perform better than experts, they don&#8217-t necessarily do a good enough job to rely on. Recently, my colleagues have started tracking the performance of one popular prediction market, at forecasting the outcome of weekly NFL games. So far, what they&#8217-re finding is that the market predictions are better than the simple rule of always betting on the home team, but only slightly so &#8212- which, oddly, is very similar to what Tetlock found regarding his experts. Some outcomes, in other words, and possibly the outcomes we care about the most, simply aren&#8217-t &#8220-predictable&#8221- in the way we would like.

–

  1. Prediction markets are not &#8220-a substitute for expert opinions&#8221-. They are a substitute for the averaged probabilistic predictions of a large group of experts polled the traditional way (by phone or by e-mail). In prediction markets, traders (who are not experts, most of the times) collect and aggregate facts and expertise at a lower cost than a poll or survey of experts.
  2. In the research cited by Ducan Watts, the prediction markets are slightly more accurate than the competitive forecasting mechanism. Well, that&#8217-s something we are used to.
  3. What Ducan Watts doesn&#8217-t say is that prediction markets integrate facts and expertise faster than the group of experts polled by his researching colleagues &#8212-for the very crude reason that it takes a certain time to survey a group of experts (be it by e-mail or by phone).

–

If I can count, that&#8217-s 3 reasons why prediction markets can bring in business value:

  1. lower cost-
  2. better accuracy (relatively, and, overall)-
  3. velocity.

That said, it should be repeated that prediction markets feed on facts and expertise &#8212-so the experts remain indispensable in the general forecasting process.

No facts (e.g., political polls) &#8211-&gt- No prediction markets.

No experts (e.g., NFL prognosticators) &#8211-&gt- No prediction markets.

–

Flu prediction markets can correct Google Flu Trends.

Posted on by
1

No Gravatar

2 practicing physicians laugh at using collective intelligence for nation-wide flu detection:

[…] Flu Trends tracks almost perfectly with data on influenzalike illnesses that the CDC obtains from doctors&#8217- offices. And as an added bonus, Flu Trends detects outbreaks up to two weeks earlier, when people are still sitting at home sneezing into their keyboards. […]

But if officials monitored only Flu Trends, it would be difficult to sort the signal from the noise —in addition to losing critical details on who is sick. Things besides an actual flu outbreak can cause people to search the Internet for flu information. We would imagine that Flu Trends would spike on the release date for a flu-related movie —maybe Outbreak 2: Electric Booga-Flu. And what happens if a pandemic flu scare hits the nightly news? Flu Trends&#8217- ability to detect when the real pandemic hits will be obliterated when people, including those without symptoms, start to search the Internet. Monitoring drugstore sales has the same issue: A jump in cold-medicine sales may mean a flu outbreak, but it could also mean that CVS is running a sale or that flu fear is causing people to stock their medicine cabinets. […]

–

They end their articles saying that Google can&#8217-t cure the flu, anyway. [???]

–

The response to the objections they jot down in the 2nd paragraph above is easy:

  • Informed by all other means, the event derivative traders can determine whether the spikes in Google Flu Trends are due to abnormalities (see the 2nd paragraph in the excerpt above) or due to the real spreading of influenza.
  • Hence, the flu prediction markets have a much higher social utility than Google Flu Trends. Chris Masse said so.
  • David Pennock, go writing another research paper about that.
  • History will retain that David Pennock was research scientist under Chris Masse&#8217-s reign in the field of prediction markets.

–

Google Flu Trends

Iowa Health Prediction Market

The “predict flu using search” study you didn’t hear about – by our good Doctor David Pennock

BBC

New York Times

WSJ Health blog

University College Cork (UCC) School of Medicine + Intrade

Dylan Evans&#8217- website

–

Previously: #1 + #2 + #3

–

The IFTF X2 Project

Posted on by
Reply

No Gravatar

[…] X2 will identify major trends and disruptions in science, technology, and the practice of science over the next twenty years and their impacts on the larger society.

X2 will utilize an open-source web platform that will help individuals and organizations track and analyze global trends in science and technology. The project will employ bottom-up forecasting methods, making use of the collective intelligence of people with different backgrounds, domains of expertise, and geographic locations to synthesize larger patterns and trends. […]

http://www.sciencex2.org/

Google vs. Prediction Markets – Which of the 2 will detect the flu, first?

Posted on by
3

No Gravatar

An Irish research team hopes to make accurate forecasts of key public health indicators.

University College Cork (UCC) School of Medicine + Intrade

Dr Dylan Evans:

Prediction markets are [specialized], small-scale financial markets operated to predict future events. The idea is that the collected knowledge of many people, each with a different perspective, will be more accurate than an individual or small group or even experts.

When they have been used to predict the outcomes of political elections, prediction markets have been found to be more accurate than alternative methods of forecasting.

The obvious area to look at in the first instance is infectious disease, but we plan to extend our research into many other areas of public health. At the moment, people do not get data on infectious disease until it&#8217-s a couple of weeks out of date and we need to get it quicker.

–

Dylan Evans&#8217- website

–

My opinion:

  • To assess the benefits (if any) of the prediction markets used as forecasting tools for public health, researchers will have to compare them with the experts&#8230- and with the &#8220-Google Flu Trends&#8221- web service, which is entirely free of charge and free of advertising (being sponsored by the Google Foundation). Does not sound good for the prediction markets.
  • The irony is that it&#8217-s our prediction market researchers (David Pennock and his accomplices) who gave weight to this non-market tool. &#8212- Pennock = Treator &#8230-!!&#8230- [ :-D – Joke. ]

–

APPENDIX:

– Iowa Health Prediction Market

– Google Flu Trends

– See also: Google Foundation on &#8220-Predict and Prevent&#8221-.

– Google Trends

– David Pennock on the fact that flu-related searches on the Web are precise predictors of the upcoming influenza outbreaks.

– BBC

– New York Times

– WSJ Health blog

–

Are collective intelligence solutions being oversold?

Posted on by
1

No Gravatar

Xpree CEO Mat Fogarty:

Chris,

Management struggles to understand and plan for the future. When forecasts are inaccurate, corporations incur huge costs due to inventory write-offs, stock-outs, misallocated resources or cost of capital. Collective intelligence delivers objective, accurate forecasts in real time, thus saving many millions of dollars for our corporate clients. The solution is not being oversold, to the contrary, the potential vastly exceeds current awareness and adoption.

&#8220-If foresight is not the whole part of management, at least it is an essential part of it&#8221- (Henri Fayol, 1916).

In my 10 years experience as a management accountant and corporate planner, I have witnessed multiple forecasts suffer from inaccuracy due to uncertainty and biases. Whether forecasting a launch date, sales volume or cost of development, it is the systematic biases due to incentive systems, politics and common cognitive errors that contribute more to inaccuracy than the uncertainty. The problem stems from the fact that the owner of a forecast is normally the owner of the business unit / sales team / project, and budgets and bonuses are based on forecasts. This necessitates game playing and politics and makes the development of an objective, accurate forecast near impossible.

Collective intelligence can overcome these problems by incentivising a diverse crowd of knowledgeable employees to share their insight, balancing the resulting estimates, and rewarding accuracy and timeliness.

However, we are at an early stage in the development of this opportunity. There is still work ahead of us to develop the ideal mechanism to combine simplicy of UI [user interface] with richness of information gathering. In addition, we need to further develop the way collective intelligence interfaces with traditional corporate structures, processes and systems. These are Xpree&#8217-s challenges&#8230- stay tuned.

Mat
CEO, Xpree

The Intrade bettors expected Mr. Obama to end up with 364 votes in the Electoral College -one less than he actually got.

Posted on by
5

No Gravatar

My remark to John Tierney:

InTrade got it [almost] spot on because they were wrong on Missouri (which was predicted to go for Obama but went to McCain) and wrong too on Indiana (which was predicted to go for McCain but went to Obama) —and those 2 opposite mistakes canceled themselves because those 2 states have the exact same number of electoral votes (11). Hence, I disagree with your method.

–

–

APPENDIX:

Here&#8217-s a visual post-mortem of the 2008 US presidential elections.

Pay attention to Missouri and Indiana.

–

A) InTrade, on November 5, 2008 (screen shot taken at 2:00 am):

–

–

Prediction Markets &amp- State Polls, on November 4, 2008:

–

–

B1) Prediction Markets (on November 4, 2008)

–

–

InTrade (screen shot taken at mid-day ET, November 4, 2008):

–

InTrade (screen shot taken in the morning, November 4, 2008):

–

BetFair (screen shot taken in the morning, November 4, 2008):

–

HubDub (screen shot taken in the morning, November 4, 2008):

–

–

–

–

–

B2) State Polls (on November 4, 2008)

–

–

Karl Rove (on November 4, 2008):

–

CNN (on November 4, 2008):

–

Pollster (on November 4, 2008):

–

Electoral-Vote.com (on November 4, 2008):

–

Nate Silver (on November 4, 2008):

–

–

PREDICTION MARKET PROBABILITIES

–

Explainer On Prediction Markets

–

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative represents the imputed perceived likelihood of the partially uncertain event (i.e., its aggregated expected probability). A 60% probability means that, in a series of events each with a 60% probability, the favored outcome is expected to occur 60 times out of 100, and the unfavored outcome is expected to occur 40 times out of 100.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism &#8212-with or without an automated market maker.

Prediction markets enable us to attain collective intelligence. Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that the traders bring when they agree on prices. The event derivative traders are informed by the primary indicators (i.e., the primary sources of information), like the polls, for instance. These informed speculators then execute their transactions based on their anticipations about the future &#8212-anticipations that will be either confirmed or infirmed.

The value of a set of prediction markets consists in the added accuracy that these prediction markets provide relative to the other forecasting mechanisms, times the value of accuracy in improved decisions, minus the cost of maintaining these prediction markets, relative to the cost of the other forecasting mechanisms. According to Robin Hanson, a highly accurate prediction market has little value if some other forecasting mechanism(s) can provide similar accuracy at a lower cost, or if very few substantial decisions are influenced by accurate forecasts on its topic.

–

More Info:

– The Best Resources On Prediction Markets = The Best External Web Links + The Best Midas Oracle Posts

– Prediction Market Science

– The Midas Oracle Explainers On Prediction Markets

– All The Midas Oracle Explainers On Prediction Markets

–

Competitive Forecasting (the brand which NewsFutures Emile Servan-Schreiber is so sanguine about) is probably more than a generic mark, it might well be a descriptive mark -provided X, Y and Z.

Posted on by
Reply

No Gravatar

Tom W. Bell – (a law professor who have been following the field of prediction markets for years):

My two cents on “competitive forecasting”: It probably rises above a merely generic mark, which could never be protected, because it is not the commonplace name for the service to which it refers. It more likely qualifies as a descriptive mark, and as such could be protected only if “secondary meaning” were proven. In other words, the claimant would have to show that by dint of long exposure to its use in a commercial context, consumers had come to understand the mark not as a mere description but as the name of the claimant’s service. Whether or not “competitive forecasting” can meet that test remains a question of fact, of course.

Caveat: I speak only of U.S. law, though most common law countries follow similar principles.

–

Interesting.

PostScriptum: Put aside that discussion about branding, I like NewsFutures as a play-money prediction exchange, and I have come to realize, e-mailing Emile privately, that he is one of the man I would go for to have an in-depth foray into the real value of the prediction markets (going beyond accuracy, onto utility) &#8212-ironically, the kind of stuff that Robin Hanson is researching more seriously these days (PPT file).

–