I’ve been running a niche prediction market site for research over the last three summers. Recently, some of these play-money markets overlapped with real-money markets currently on-going on Betfair.

In my post over at Mercury’s Blog, I discuss how I’ve used play-money market wisdom to take advantage of some poor market-makers on the real-money Betfair markets. Specifically, the favourite on one particular play-money market has a probability to win of ~80% (which I think is fairly accurate). I managed to buy that same contract on Betfair at the equivalent of a 40% probability. Similar examples still exist because of market makers that skewed initial odds towards long-shots (at least by play-money market standards).

Perhaps this will strike some conversation on potential arbitrage between play-money and real-money prediction markets.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + People + Write A Post Or A Page + How To Publish

• Forrest Nelson valids Emile Servan-Schreiber.
• Subsidizing real-money prediction markets and real-money conditional prediction markets
• Arbitrage in the InTrade Dem VP Market
• Will the CFTC agree to license and regulate real-money prediction markets?
• 2 MILLION TRADES LATER: Inkling’s play-money prediction markets are accurate —too.

Over on Mercury’s Blog, I’ve been posting a series of draft responses to the specific questions raised by the CFTC in the Concept Release. In order to prevent a far-too-long single post, I’ve separated them into five sections:

• Intro and Broad Overview questions
• Public Interest
• Jurisdictional Determination
• Legal Implementation
• Market Participants

Overall, I believe that prediction markets / event contracts should be opened as widely as possible with as little regulation as possible. That said, retail customers in a real-money prediction market should have sufficient protections against poorly run marketplaces.

I believe that the safe harbour provisions that the very esteemed group of academics promote are a good start, but do not go far enough. Those provisions are a very good way to deal with the markets they discuss: academic, internal corporate or small-stakes (a la IEM) markets.

In my drafts, I write that the CFTC should have jurisdiction for event markets, that retail event markets should be allowed, and they should be allowed with as little regulation as possible while keeping consumers protected. This last balance may be difficult to find at first. What I suggest is that the CFTC allow individual marketplaces to operate providing that the marketplace operator self-certifies that:

1. The rules of the contract are specifically stated for each contract
2. The marketplace has a dispute resolution procedure
3. The marketplace provides a basic warning to retail customers that they can lose their investment

Tom Bell, in reading some of my early drafts, doesn’t like what I’ve said in reference to contracts on assassinations, terrorist attacks, etc. I believe that any contract that is based on a criminal act (as defined by US law) should not be allowed. My reasoning is two-fold: I don’t believe that any contract should exist that provides an incentive to break laws, and I believe that the public in general would find a market for these contracts “morally repugnant” (to use an Al Roth phrase).

Tom believes that the CFTC should not forbid trading in these claims because they would offer thin and traceable trading. Perhaps part of my difference with him is in these assumptions; I don’t believe the CFTC should be regulating event marketplaces so tightly that trading would be 100% traceable. To do so would force exchanges to verify customers identities and take other steps that I personally believe are too burdensome in a regulatory sense.

That said, I don’t believe that all of these markets should be forbidden, just those that would violate US laws. Markets on foreign terrorism (such as terrorist bombings in international “hot spots”) would be allowed, and provide potentially useful information. Whether these markets should be offered at all would still be up to each exchange, based on the demand from that exchange’s customers.

Summary

Please do browse through my responses. Perhaps something will spark discussion and lead to a better overall reply to the CFTC.

I would appreciate any and all comments; you can do so directly on each post. And if you would like to co-sign what I plan to submit, please feel free to contact me directly.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + People + Write A Post Or A Page + How To Publish

• My response to the CFTC on event contracts
• The CFTC is going to close the comments in 3 days. We have 3 days left to convince the CFTC to accept FOR-PROFIT prediction exchanges (e.g., InTrade USA or BetFair USA), and counter the puritan and sterile petition organized by the American Enterprise Institute (which has on its payroll Paul Wolfowitz, the bright masterminder of the Iraq war).
• Let Prediction Markets Fight Terrorism.
• Chris Masse’s first comment to the CFTC on “event markets” (prediction markets)
• Why Robin Hanson is right to freak out about the upcoming CFTC ruling on “event markets” (prediction markets)

McKinsey & Company, the prestigious consulting firm, recently published an interview focusing on prediction markets in their McKinsey Quarterly journal. (Link to abstract is here, free registration required to read the article.)

The interview included Bo Cowgill of Google, Jeff Severts of Best Buy, James Surowiecki (author of the Wisdom of Crowds) and Todd Henderson (an ex-McKinsey consultant.) It’s an interesting eleven-page article that discusses both prediction markets in general as well as specific experiences from Google and Best Buy.

I’ve written up a full review/summary of the article on Mercury’s blog here.

This article in the McKinsey Quarterly is yet another good “traditional” source that should help encourage more companies to view prediction markets as a mainstream business tool.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + People + Write A Post Or A Page + How To Publish

• McKinsey: The Promise Of Enterprise Prediction Markets
• The Promise Of Enterprise Prediction Markets — The McKinsey conference should have been rooted in the economic science and McKinsey should have invited economists.
• Do Google’s enterprise prediction markets work? — Using Prediction Markets to Track Information Flows: Evidence from Google
• Information Sharing at Google via Enterprise Prediction Markets
• A picture worth 10,979 words?

While Chris has a good resource for prediction market software here with lots of technical details on different packages and platforms, I thought it would be useful to review it and talk about the some of the practical aspects of the different software solutions. I also wanted to stress the prediction markets as used by corporations and the public, and not include some of the general trading platforms that Chris devotes time to on his page. There are also some companies that have been doing some great projects that don’t have much detail there.

The result was a multi-page essay/review on Mercury’s Blog. It’s long enough that I don’t want to cross-post the entire thing, but instead wanted to post a link if you’re interested. I believe it’s the most comprehensive online resource to help companies understand the different companies and software solutions available in the prediction market industry.

Click below to check it out:

A (long) review of prediction market software

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + People + Write A Post Or A Page + How To Publish

• Delivering… Software as a Service - SaaS
• NewsFutures’ hyper marketese on their prediction market consultancy and software package for enterprise prediction markets
• Google’s Bo Cowgill takes a swipe at the prediction market software vendors.
• How to make a set of Xpree prediction markets in 3 easy steps
• REBUTTAL: SalesForce, StarBucks and Dell demonstrate that enterprise prediction markets as intra-corporation communication tools (as opposed to forecasting tools) are overhyped by the prediction market software vendors and a little clique of uncritical courtisans.

[Cross-posted from Mercury's Blog. Click here to subscribe by e-mail.]

I’ve recently read about a very unique software product, developed by a company in New York City. That product is called FogBugz 6.0, and was written by Fog Creek Software. The company is famous for the blog started by its founder, Joel Sposky, called “Joel on Software.”

FogBugz 6.0 is a software bug tracking and project management program that uses an innovative approach to forecasting ship dates based on what they are calling “Evidence Based Scheduling (EBS).” I think this software is likely very excellent at what it does. I also think that this type of feature is the perfect foil for how and why prediction markets can be used to forecast project management dates in companies and other organisations.

Prediction Markets and Evidence Based Scheduling provide the exact same core forecast data. A probability on when a project will be completed, and a plot showing how that probability has changed over time. Where they differ is in approach: bottom-up versus holistic.

Bottom-up approach:

Evidence Based Scheduling is a very data-centric approach. Each developer tracks their estimate to complete a given project segment. The software then provides a timer to measure exactly how much time was actually spent completing that segment. With that data in hand, the time it will actually take for a given developer to complete future segments can be calculated, with individualised standard deviations.

What the software requires is a list of project segments and who they’re assigned to. Monte Carlo simulations sum them all together to create the final probability curve.

What’s great about it: It’s data-based, and managers can dive down into the data to see why (and who) is the reason that a project may or may not make it out the door in time. Evidence Based Scheduling is also automatic, and doesn’t require any other interaction from the user to develop the forecasts. In a very data-rich task environment, this could be an excellent tool.

The problems with it: This works particularly well within its specific context, but I am skeptical that this would work elsewhere. Here are my concerns:

• Poor data / Garbage In=Garbage Out: The reason this implementation works is because each task is tracked and timed individually. The moment that the discipline around the process is lost, the data quality will suffer, and that will directly impact the forecast.
• Scope Creep: This method is based on having all tasks detailed. If a project suffers from scope creep, where additional features are consistently added throughout, the forecast is essentially meaningless. There are plots within this EBS to attempt to show this phenomenon, but the fundamental issue is still there.
• Can it deal with complexity?: I’m not convinced that EBS will deal well with projects that have lots of moving parts, or significant internal political issues. Political issues in particular can take completely de-rail well-run projects, and there is nothing in EBS to account for these complexities. (This is a bit of a combination of the two points above.)

Holistic approach:

Prediction Markets are another way to forecast when projects will key milestones. This can be done in a variety of ways, depending on your business needs. (You can find out more on prediction markets by watching these short videos: What is a prediction market? and How can I use a prediction market in my business?)

What’s great about it: Prediction Markets capture the holistic picture of a project. Instead of trying to forecast base-level data and then sum each part up, prediction markets forecast what you really want to know. Incentives encourage each user to express what they’re thinking, and their knowledge and history with similar problems at your company and in your industry. Even better, their level of activity corresponds to the depth of their conviction; if someone doesn’t feel they know enough they won’t trade heavily and those with heart-felt conviction will trade significantly more; it’s self-selecting.

The problems with it: Compared to Evidence-based Scheduling, prediction markets do have some issues. These are where prediction markets don’t do as well:

• No audit trail. While prediction markets provide forecasts, managers and executives can’t necessarily dive down into the data to see why they are changing. This can be mitigated by adding forums/discussion boards to the market, or providing a capacity to ask people why they traded after making a transaction.
• Less scientific “feel”. Evidence-based scheduling looks great because you can see and understand the building blocks of how forecasts are built. Prediction markets are based on people, their information and their incentives. Despite the fact they’ve been proven to work better than other forecasting methods, the “science” of prediction markets is based in economics, not in statistics.
• More work is required. Employees need to spend a few minutes of their time throughout the week/month in order to trade. EBS simply runs in the background (other than making the initial forecast of how long it will take to complete a segment of work). Time spent on a prediction market site can also be seen as a distraction by some managers, despite the valuable feedback it provides them.

Summary:

Evidence-Based Scheduling is a fantastic tool in its context; I just believe there aren’t many business problems where it can be effectively used. Prediction Markets use a holistic method to look at a project in total. While there isn’t as strong an audit trail in a prediction market, the full scope of the problem in the context of the company and the industry is taken into account. For many organisations, a prediction market will be the optimal solution.

Screenshot from FogBugz 6.0 website.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + People + Write A Post Or A Page + How To Publish

• Does the field of forecasting encompass prediction markets?
• The Folly of Forecasting - by Barry Ritholtz
• Demand forecasting systems: Spending a lot on software doesn’t guarantee success.
• NewsFutures’ Competitive Forecasting and Idea Pageant technologies are both featured on UC Riverside’s eLab eXchange
• Two vivid British teens give you a tour of BetFair.

In the past month, there have been two excellent conferences on prediction markets. Consensus Point hosted a conference in New York City on September 24th, and another conference was held in London on October 11-12th.

I took some fairly extensive notes during both conferences, and the links are below.

New York:

Highlights: James Surowiecki, preview of Best Buy’s prediction market project, ConsensusPoint’s new trading interface.

Part 1 - Robin Hanson, Dave Perry (ConsensusPoint), Misys, My Currency
Part 2 - Media Predict, Qualcomm, GE Global Research, CNBC, (Panel Discussion), Mercury Research & Consulting, James Surowiecki

London:

Highlights: Demo of Microsoft’s prediction market software, stories of prediction market successes from Nosco and Pro:kons, John Delaney of InTrade answering probing questions.

Part 1 - Gexid, InTrade, Xpree, Prof. Leighton Vaughn Williams
Part 2 - FT.com, Newsfutures, Microsoft (Information Markets Group), Nosco
Part 3 - Mercury Research & Consulting, Prof. Peter Sorensen, Pro:kons

It’s pretty difficult to set up a webcast, so until conferences are able to do that more easily (as I know Chris would like), I hope these notes are useful for those that weren’t able to attend. Click here to contact me if you have any questions.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + People + Write A Post Or A Page + How To Publish

• Robin Hanson to speak at the 2007 Prediction Markets Conference
• Prediction market sessions of the O’Reilly Money-Tech Conference suffer fatally from the absence of the world’s most knowledgeable, most innovative and most trustworthy prediction market expert.
• Prediction Markets Conference in New York City
• Reminder: Corporate Applications of Prediction Markets Conference (1 November)
• Bloggers will provide the best coverage on Earth for your prediction market conference(s).

A new video has been posted to YouTube (and embedded below) which explains “How can I use a prediction market in my business?”. As before, the answer is a presentation in three-minutes or less.

The first video in the series, “What is a prediction market?”, has been viewed nearly 200 times on YouTube. I was pleased with the responses I received on this blog and via e-mail. Thank you to those of you who took the time to provide feedback; it was very valuable.

The purpose of the video below was to break down the different types of ways prediction markets can be used in a company. Again, it’s fairly high-level, but approaches it from the perspective of someone that has never encountered prediction markets before.

I look forward to reading your comments and e-mails about this video! As always, click here to e-mail me with any questions about your prediction markets project, or with any feedback.

Cross-posted from Mercury’s Blog.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + People + Write A Post Or A Page + How To Publish

• A prediction markets explainer
• Jed Christiansen’s video explainer on prediction markets
• CNBC airs an upbeat explainer about InTrade’s prediction markets.
• ABC 20/20 — A good (but servile) explainer on the wisdom of crowds and the prediction markets
• VIDEO — Bo Cowgill on Google’s enterprise prediction markets — O’Reilly Money:Tech

A few weeks ago, Chris Masse complained that the everyday layperson doesn’t get prediction markets. I found it a bit amusing, as I had just started work on a presentation to help fix just that.

The goal I set was to explain prediction markets to a layperson in three minutes or less. That meant that much of the finer details weren’t addressed, but I was able to address the core concepts. Today I posted that video on Mercury’s Blog, and you can see it yourself by clicking on this link.

I look forward to your comments on the video. Depending on the reaction from this video, I plan to do a series of similar videos to go into further detail on various aspects of prediction markets, all in three minutes or less. (The web is all about short attention spans, these days.)

UPDATE: I’ve managed to get the video up on YouTube properly.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + People + Write A Post Or A Page + How To Publish

• A new prediction markets explainer video
• Jed Christiansen’s video explainer on prediction markets
• CNBC airs an upbeat explainer about InTrade’s prediction markets.
• Storage Markets explainer on prediction markets
• ABC 20/20 — A good (but servile) explainer on the wisdom of crowds and the prediction markets

When pitching a prediction market forecasting solution within a company, everyone from the consultant like myself to the software vendor to the internal company champion needs to recognise the real value of hosting prediction markets inside the organisation. The value to the business is NOT that prediction markets are accurate. Let me repeat that: the value to the business is NOT that prediction markets are accurate.

Why is that? Well, it’s because accuracy is a bad story. If you start talking accuracy, then you need to discuss calibration, scoring rules, and the like. This gets complicated fairly fast, particularly when you forecast probabilities. (David Pennock has a good run-down of the topic here if you want to read further.) The real reason that accuracy isn’t the primary business value is because increased accuracy simply isn’t that valuable to executives. Forecasting has always been imperfect because the future can never be perfectly known, and an improvement here is still imperfect.

So what is the value of prediction markets in an organisation? I would recommend listening to Todd Proebsting talk about the first business-related prediction market that Microsoft held. (Link here is to yahoo confab conference on prediction markets.) His story described a prediction market that was very accurate. But the real power of his prediction market story was that it revealed undeniable evidence to the project leadership that there were problems with their product. Using a prediction market connected the leadership of the project with their team. Without having to read reports, talk with scores of employees, or other time-intensive activities, the project leadership was able to immediately receive feedback from the team on a key metric: when the project was going to be completed.

This is what I try to impress upon companies that are looking into prediction markets: they are tools for connecting with employees throughout an organisation. In the best case scenario, the prediction market forecast is exactly what the project manager / executive expects, or it matches up with previous forecasting methods. In the best case nothing has been lost (except a marginal cost for operating a market), and the company has gained by interacting with their employees and fostering another type of community at the company.

In the alternate scenario, the prediction market forecast is dramatically different than what the executives believe, or what other forecasting models show. This case is where prediction markets hold their true value: in forecasting surprises. If a company’s leadership can be informed of potential surprises with sufficient time, they can take action to mitigate or solve problems. Though the managers in Todd’s example probably didn’t like being told of a surprise, in the end I’m sure they appreciated being able to manage the problem proactively.

• How much is it worth to your company to realise six months ahead of time that the project isn’t going to finish on time?
• How much is it worth to your company to realise that if the new product’s quality will be poor?
• How much is it worth to your company to realise that your employees believe that the new product won’t get nearly the number of sales that you’re counting on?

I’d like to repeat that the value of prediction markets to a business is NOT their accuracy. The value of prediction markets to a business is that they quickly and efficiently connect company and project leaders with their employees and teams to get insights on important issues.

—————

Quick notes: if you have QuickTime Pro, if you click on the link above you should be able to download the two and a half hours of conference as a QuickTime movie. It’s around 590 MB, though, so it will take some time! I’m working on burning a DVD from the MP-4, please contact me if you’re interested.

Finally, if you like reading this blog, please click this link to subscribe via e-mail, or click this link to subscribe via an RSS reader.

Cross-posted from Mercury’s Blog.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + People + Write A Post Or A Page + How To Publish

• INTEL BUSINESS CASE: INTERNAL PREDICTION MARKETS DO WORK.
• A Ten-Year Research Project Investigating Business-to-Business Reverse Auctions
• Small Business = GOOD — Big Business = BAD
• Inkling Markets = Business Processes + Prediction Markets
• INTEL BUSINESS CASE: Does Intel really use internal prediction markets?

Writing effective contracts can be very difficult. Chris Masse has blogged about the InTrade/TradeSports problems closing out the markets regarding North Korea launching missiles, as well as some issues regarding BetFair and control of the US Senate after the 2006 elections, all of which occurred because of poorly designed contract settlements.

I’ve created four different markets on Inkling to test a different type of contract and settlement criteria. What many people are interested in for each Congress are:

1. Which party will control the Senate and House of Representatives?
2. How many seats will each party have in the Senate and House of Representatives?

More people are generally interested in the “control” question than the “number of seats” question, probably because it takes much more knowledge and research to effectively trade on the number of seats each party will have. Unfortunately, this can cause problems depending on how the contracts were written.

How do you define control of the Senate or House of Representatives? Do you define it by a majority of seats? Or do you define it by a plurality of seats? The results may be dramatically affected in a close election when third party candidates are involved. After the 2006 elections, who would you say controlled the US Senate? There are currently 49 Democrats and 49 Republicans; with the Vice-President, it could be argued that the Republicans controlled the Senate. Unfortunately for Republicans, the two Independents caucus with the Democrats, so in reality the Democrats control the Senate.

What I have done in the Inkling markets (the House market, and the Senate market) is to base the settlement of the markets off of which party elects a key leader of each chamber. The statement I used for the Senate market is here:

[Party] will control a majority of the seats in the Senate after the 2008 election. If a third party (or independents) prevents either the Democrats or Republicans from an outright majority of seats, this market will settle based on the party membership of the President Pro Tempore.

This covers both scenarios: the much easier scenario where a party has a clear majority of seats thus winning control, and the scenario where a coalition results. Given the nature of the two parties in Washington today and the election procedures of the US Senate, the President Pro Tempore will certainly be a member of either the Democratic or Republican party. I used the same type of settlement criteria in the House of Representatives market, basing it on the party membership of the Speaker of the House. I believe this is an effective way to determine which party will really control each chamber, since it goes beyond the mechanics of getting control (number of seats and such) and speaks to the party that actually has the power in each chamber (who has the authority).

The other type of market (how many seats will each party have) is quite a bit simpler. In the US Senate, with 100 seats, I created markets for the Democrats, Republicans, and Third Party candidates. In the US House of Representatives, I decided to use a market based on percentage of seats instead of the exact number of seats. This is because I wanted to use the Inkling market which forced all of the options to sum to 100%. Unfortunately, this scaled claim may be a bit confusing to traders, but I tried to explain it with this statement:

[Party] will control this percentage of seats after the 2008 elections. Each percentage point corresponds to 4.35 seats. (Alternately, each seat corresponds to 0.23 percentage points.)

While in a perfect world I could have created a market that automatically summed to 435 instead of 100, this still works to elicit the information I’m interested in for the 2008 elections.

Elections are like many business problems; writing contracts that are based on what your organisation really wants to know can be difficult. I hope that this post provokes some different ways of looking at potential prediction markets in your company. Please contact me with any questions.

Cross-posted from Mercury’s Blog

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + People + Write A Post Or A Page + How To Publish

• BetFair: “Which of these parties will have more seats in the US Senate following the 2006 US Senate Elections?”
• Google’s Bo Cowgill: Organizations experimenting with prediction markets don’t shy away from publicizing it.
• Big firms experimenting with internal prediction markets as IAM
• Forecasting House Seats from Generic Congressional Polls
• 2006 North Korea Missile prediction market: a scandal signed TradeSports-InTrade