Proving business value

Jed Christiansen May 22nd, 2007

When pitching a prediction market forecasting solution within a company, everyone from the consultant like myself to the software vendor to the internal company champion needs to recognise the real value of hosting prediction markets inside the organisation. The value to the business is NOT that prediction markets are accurate. Let me repeat that: the value to the business is NOT that prediction markets are accurate.

Why is that? Well, it’s because accuracy is a bad story. If you start talking accuracy, then you need to discuss calibration, scoring rules, and the like. This gets complicated fairly fast, particularly when you forecast probabilities. (David Pennock has a good run-down of the topic here if you want to read further.) The real reason that accuracy isn’t the primary business value is because increased accuracy simply isn’t that valuable to executives. Forecasting has always been imperfect because the future can never be perfectly known, and an improvement here is still imperfect.

So what is the value of prediction markets in an organisation? I would recommend listening to Todd Proebsting talk about the first business-related prediction market that Microsoft held. (Link here is to yahoo confab conference on prediction markets.) His story described a prediction market that was very accurate. But the real power of his prediction market story was that it revealed undeniable evidence to the project leadership that there were problems with their product. Using a prediction market connected the leadership of the project with their team. Without having to read reports, talk with scores of employees, or other time-intensive activities, the project leadership was able to immediately receive feedback from the team on a key metric: when the project was going to be completed.

This is what I try to impress upon companies that are looking into prediction markets: they are tools for connecting with employees throughout an organisation. In the best case scenario, the prediction market forecast is exactly what the project manager / executive expects, or it matches up with previous forecasting methods. In the best case nothing has been lost (except a marginal cost for operating a market), and the company has gained by interacting with their employees and fostering another type of community at the company.

In the alternate scenario, the prediction market forecast is dramatically different than what the executives believe, or what other forecasting models show. This case is where prediction markets hold their true value: in forecasting surprises. If a company’s leadership can be informed of potential surprises with sufficient time, they can take action to mitigate or solve problems. Though the managers in Todd’s example probably didn’t like being told of a surprise, in the end I’m sure they appreciated being able to manage the problem proactively.

How much is it worth to your company to realise six months ahead of time that the project isn’t going to finish on time?
How much is it worth to your company to realise that if the new product’s quality will be poor?
How much is it worth to your company to realise that your employees believe that the new product won’t get nearly the number of sales that you’re counting on?

I’d like to repeat that the value of prediction markets to a business is NOT their accuracy. The value of prediction markets to a business is that they quickly and efficiently connect company and project leaders with their employees and teams to get insights on important issues.

—————

Quick notes: if you have QuickTime Pro, if you click on the link above you should be able to download the two and a half hours of conference as a QuickTime movie. It’s around 590 MB, though, so it will take some time! I’m working on burning a DVD from the MP-4, please contact me if you’re interested.

Finally, if you like reading this blog, please click this link to subscribe via e-mail, or click this link to subscribe via an RSS reader.

Cross-posted from Mercury’s Blog.