Download this post to watch the video —if your feed reader does not show it to you.

External Links:

- CrowdCast

- CFO Prediction Market

- Matt Fogarty’s twitter – [Midas Oracle's twitter]

Previously:

- CrowdCast = Collective Forecasting = Collective Intelligence That Predicts

- CrowdCast = market mechanism = binary spreads with a market maker

It has been suggested that EPMs have helped Best Buy getting it right on the ‘HD-DVD versus Blu-Ray’ issue. It’s a boatload of bullsh*t. I know a lot about technology intelligence. It should be done by a smart and curious operator. There is no need of enterprise prediction markets to do this task. The tools you need consist of a bunch of IT news aggregators and a good search engine. Consider this:

The Inevitable Move Of iTunes To The Cloud

In the ‘cloud’ piece above, there are facts and there are speculations. You’ve got much more technology intelligence reading the ‘cloud’ piece above than you would get from a crude, plain and simple prediction market. Gimme a break with EPMs. Make no sense at all.

Contrast EPMs (which are costly) with public prediction markets (a la InTrade or BetFair), where probabilistic predictions are offered for free. That makes all the difference for the reason that the added accuracy brought by prediction markets is very small. Market-generated odds are handed out for free to journalists —still, few of them take the bait. The market-powered crystal ball is worth peanuts.

The reason CEOs are paid millions is that only a small percent of the population of business administration managers has the ability to cut through the non-sense and the balls to cut the cost of the non-sense. It is a rare skill. I am calling on CEOs to end EPMs. Right now.

My review of the literature and case studies (that have been published) indicates that prediction markets have improved the accuracy of forecasts, but the improvements have not been great enough to encourage widespread (or even minimal) acceptance. Furthermore, these studies like to average their results over a number of markets, disguising the fact that some markets improve forecasts, while others fail to do so. Some studies look at average absolute errors, covering up the fact that some predictions were underestimating the true outcome and others overestimating it. This means the real errors are as much as twice as large as those reported. Few, if any, explanations for the failures are ever presented. This raises the issue of consistency. In case studies such as these, where there is no clear under- or over-estimation tendency, for which a correction may be made, the prediction errors are just too great.

Clearly, if similar prediction markets do not provide consistently accurate forecasts, they will not be relied upon for any important business decisions.

I warned my readers many times against the vendor conferences organized by the San Francisco man. He is so desperate that he invites anybody who will pronounce the word “prediction” and “markets” in the same paragraph. Many of the invited speakers haven’t the slightest knowledge of the field of prediction markets. As for the vendors, they are incapable of producing one single case study featuring a success in the use of enterprise prediction markets. Not a single one. (And I won’t mention the “flow of information” —the worst research ever published on prediction markets.) Their vendor websites publish lists of clients, which, at first glance, look impressive, but many of those so-called customers are in fact ancient clients who have ended pilot programs years ago. To add insult to injury, this fake conference is sold $400 to gullible attendees. It is not even worth 4 cents.

The Economist reporter who attended the San Francisco conference realized what I [*] realized long ago: The field of enterprise prediction markets is all smokes and mirrors. The more the prediction market vendors will participate in such crappy conferences, the more the media will realize that the prediction market vendors are all hat and no cattle, and the more they will publish news stories bursting the prediction market bubble. And in the end of 2009, we will end up with 10 news articles in major media telling the world that prediction markets were a fad. Live by the hype; die by the hype.

The only way to get out of this debacle is to come back to basics: Do the research right, do discover the real value of enterprise prediction markets (velocity), and, then, only when you have something to show for it, go out in postings and conferences.

-

[*] I follow the field of prediction markets since 2003. I saw it in all shapes and stripes. You can fool your mother, but you can’t fool me.

-

APPENDIX:

An uncertain future – A novel way of generating forecasts has yet to take off. – by The Economist – 2009-02-26

- But although they have spread beyond early-adopting companies in the technology industry, they have still not become mainstream management tools. Even fervent advocates admit much remains to be done to convince sceptical managers of their value.

- Koch says the results so far have been pretty accurate compared to actual outcomes, but stresses that markets are complementary to other forecasting techniques, not a substitute for them.

- A big hurdle facing managers using prediction markets is getting enough people to keep trading after the novelty has worn off.

- Another reason prediction markets flop is that employees cannot see how the results are used, so they lose interest.

- Bosses may also be wary of relying on the judgments of non-experts.

-

- But although they have spread beyond early-adopting companies in the technology industry, they have still not become mainstream management tools. Even fervent advocates admit much remains to be done to convince sceptical managers of their value.

- Koch says the results so far have been pretty accurate compared to actual outcomes, but stresses that markets are complementary to other forecasting techniques, not a substitute for them.

- A big hurdle facing managers using prediction markets is getting enough people to keep trading after the novelty has worn off.

- Another reason prediction markets flop is that employees cannot see how the results are used, so they lose interest.

- Bosses may also be wary of relying on the judgments of non-experts.

My remarks:

1. The recent San Francisco conference (where the reporter from The Economist got his/her tidbits) has been counter-productive. No wonder, since that conference has no editorial line (it is a vendor conference). I told you so.
2. The hype about (enterprise and public) prediction markets is not substantiated. I told you so.

-

First, every market price is a prediction. Think of a familiar securities market such as a stock market. The price of a company’s stock is a forecast of the value of future dividend payments. A bond price is a forecast of the value of a defined set of interest payments, based on factors such as likelihood of default and future inflation. Second, markets generate forecasts in a very specific way – by aggregating and consolidating information from many individuals, often widely dispersed, each with access to small, idiosyncratic bits of relevant information.

This informational structure is very common in organizational life. Information within firms is often widely dispersed and undocumented, residing in the minds of employees. Junior level workers, for example, while perhaps knowing little about the overall set of strategic issues affecting their company, often have detailed understandings of isolated aspects of the business.

The fundamental challenges of corporate forecasting are to access and coordinate all relevant bits of information dispersed throughout a company and to consolidate them into a set of quantitative metrics that can be employed as forecasts.

But organizations impose significant constraints on the flow and processing of information. The hierarchy that defines organizational life often restricts the movement of information, from the bottom-up as well as across business units, and sometimes, because of various forms of “politics,” motivates the concealing of information or even the spreading of disinformation. When combined with well-documented effects such as human limitations in expressing complex thoughts and systematic biases in group decision-making, the result is that employees often do not reveal their honest assessments, sometimes because they’re not provided the opportunity and sometimes because they fear reprisal for offering an unpopular opinion. Forecast quality suffers.

Prediction markets offer firms the opportunity to incorporate the information aggregating and predictive power of markets within corporate structures relying primarily on top-down direction. A prediction market is established within a company to generate predictions on issues of interest to managers in a manner that directly addresses the foundational communication constraints within firms.

A “stock” is defined to reflect an issue of interest to managers, perhaps unit sales of a product over a specified future time period. A group of employees – perhaps salespeople and marketing personnel -are selected to participate as traders on the basis of their perceived understanding of future sales prospects. Using software that is commercially available and run as an internet (or intranet) application, the participating employees are provided trading accounts, the stock is assigned an initial value (perhaps reflecting management’s current expectation of sales in the defined period) and a currency is established to provide a medium for exchange.

With the protection of anonymity (eliminating the fear of reprisals for offering unpopular opinions) and a well-defined incentive structure, employees are motivated to acquire relevant information and contribute their best assessments. They buy and sell shares of the security based on their beliefs about future sales prospects and their desire to increase the value of their portfolio. When an employee, for example, observes that the price of the stock is less (or more) than his/her expectation of future sales, he/she will buy (or sell) the stock, thereby driving its price up (or down).

As a result of this dynamic, the stock price serves as an ongoing real-time forecast of future sales. It continuously reflects traders’ aggregated assessment of future sales of the product, in the same way that the trading of a company’s stock on a stock exchange continuously reflects the trading community’s collective assessment of the value of the company.

Several internet-based prediction markets have been functioning for many years, and many companies have implemented prediction markets internally. Performance comparisons reveal that such markets produce forecasts that are more accurate than those from traditional systems.

Prediction markets not only produce forecasts and assessments that are, on average, more accurate than those produced from traditional forecasting approaches at any point in time (because they incorporate more information and less disinformation), but also, because the markets function continuously, will reveal the impacts of new information far faster than any alternative approach. Because the usual disincentives for employees to reveal bad news to managers have been eliminated, this system can in some instances serve as an effective “early warning system.”

The informational content of a prediction market is not limited to the stock price. The underlying bid data can be examined for insights into the knowledge and the beliefs of specific employees and groups within the organization. Analysis of market transactions in prediction markets will identify areas where there is substantial disagreement among employees about future values of key parameters driving the firm’s strategic decisions. Such disagreement, reflecting a collective uncertainty about underlying factual premises and/or interpretations, will highlight areas where the incremental value of additional managerial attention, in the form of information gathering (including perhaps discussion with select employees) and/or analysis, will be particularly high.

There are additional benefits of prediction markets – such as improved decision-making on personnel issues and improved employee morale – that can be realized with the most force when the markets are employed for long time horizons.

-

Thanks to David Perry of Consensus Point for allowing me to republish this explainer.

-