Sometimes, a picture is worth 10,979 words across 33 pages. This New York Times article on corporate prediction markets doesn’t mention our implementation at Google, but the accompanying graphic (below) represents what Justin Wolfers, Eric Zitzewitz and I did better than anything I’ve seen (including our own fairly cool graphic).

No company mentioned in the article (or anywhere else, to my knowledge) is actually doing what is depicted in the picture besides Google.

Notice that the manager of the market (above) can see much, much more than current prices and trading volume. He can see who bets how how and where they sit. He can compare how specific teams or social groups are betting and whether managers for a project are betting more optimistically than their employees.

And so on. The view is a lot more interesting when you’re looking at the market like this instead of like this.

Cross-posted from Bo Cowgill.com.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + Network + Write A Post Or A Page + How To Publish

• THE GOOGLE MICRO-GEOGRAPHY PICTURE THAT WAS POSTED AND THEN DELETED BY THE PORTFOLIO MAGAZINE’S BLOGGER
• The New York Times article doesn’t mention Google’s enterprise prediction markets, alas. — Bo Cowgill says that the illustration published in the sidebar defines exclusively what is done at Google.
• At Google, employees are encouraged to place bets on an enterprise prediction exchange, which tries to forecast events based on the money wagered on a particular outcome.
• Micro-geography analysis of trading in Google’s enterprise prediction markets
• Information Sharing at Google via Enterprise Prediction Markets

The recent work on the intersection between text mining and prices counts as some of the most interesting work being done in the prediction market field. A few thoughts:

1. The concept has clear applications outside of political markets: It could be used as a trading strategy (for example, see how traders in real markets are already using similar text mining strategies) or as a market maker for illiquid markets.
2. There are applications for corporate prediction markets as well: Every day, volumes of easily accessible text are produced about companies. This comes in the firm of news articles and analysis by third parties, traffic on internal company email lists, internal and external blog posts, wikis and other available firm documents. As I wrote in July, a shrewed text mining algorithm could turn these writings into liquidity for the internal prediction markets. It’d be a bit of a problem for the current generation of prediction market vendors, who might have trouble gaining access to all these documents.
3. In either case, traders or market makers using text-mining for algorithmic trading need to deal with potential spammers who could write articles to trick the algorithm into submitting dumb bids.

On that note, the authors should consider text-mining political blogs in search of a better algorithm for predicting price movements. Firms such as Collective Intellect or Neilsen’s BlogPulse already have a large store of blog-mining data. Note that Collective Intellect is already designed to service the financial industry with data from text-mining blogs.

For text-mining junkies, Text Mining, Visualization and Social Media is a great blog.

Cross posted at my website.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + Network + Write A Post Or A Page + How To Publish

• Chris Masse’s first comment to the CFTC on “event markets” (prediction markets)
• Reading the Markets - Forecasting Prediction Markets By News Content Analysis
• Protecting Private Prediction Markets
• Nokia’s Enterprise Prediction Markets = Competitive Advantage
• Prediction Markets as Content, Part 2

Congrats to Brent Stinski and the Inkling guys on launching Media Predict and Project Publish. NYT picked up the launch today, but doesn’t mention Inkling.

Based on the article and my browsing the site, my initial reactions are:

1. Notice the words “Patent Pending” at the bottom. What could they possibly be patenting? One hopes they are not trying to create a bogus “business method” patent on the concept of prediction markets for publishing. Assuming they’re using Inkling’s standard technology, I don’t see much to patent here (granted, I haven’t logged in yet because they’re undergoing server maintenance).
2. Justin’s NYT quote about the beauty contest seems to be on-point.
3. Nonetheless, nice to see another high-profile company take interest in markets for organizational decision making.

I presume Media Predict is the high volume application that required an extra Inkling engineer. Anyone know who the hire was? I can’t seem to find the post anymore, so maybe the need went away.

Cross-posted from Bo Cowgill.com.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + Network + Write A Post Or A Page + How To Publish

• Brent Stinski’s patents for Media Predict
• The Media Predict startup is smarter than the Inkling-Giberson conglomerate.
• Media coverage on Media Predict
• Project Publish = Media Predict + Simon & Schuster
• Media Predict and “Infinite Reflection”

THE INKLING GUYS ARE RIGHT about the future of corporate prediction markets:

We’ve always felt this how [corporate prediction markets] may evolve: Instead of [..] a standalone marketplace, we expect to be fully integrated instead with an ERP system like SAP or Oracle, CRM systems like salesforce.com, or other mission critical systems of the same ilk. [I]f you think about it, the business processes prediction markets have seen success (financial forecasting, project management, product performance, competitive analysis, etc.) are many of the same inputs used in these systems. [Edits mine]

I completely agree. A corporate prediction market that talks or integrates with other corporate services confers tons of benefits. Among them:

• The market can talk with business intelligence software, corporate datawarehouses or document indexes to show relevant graphs, tables or files to traders. Better informed traders will improve predictions.
• The market can talk to HR databases to authenticate users and ensure that no employee signs up twice. In a large company, the HR databases can also help the market group employees into teams to facilitate competition among or within smaller populations.
  • Don’t forget: One day, the dream of employee bonuses for good trading might come true. On that day, the market will need to talk to the HR and compensation systems.
• Integrated markets can display where managers and employees are already looking. No need for an intra-firm marketing push to get people visiting the trading site. Placing the markets there also confers authority and legitimacy to the practice.

I could go on about this potential. The interesting question is: What does this mean for companies selling prediction market software?

1. Does this require the markets to be hosted completely within the firm? My impression is that the current generation of corporate prediction markets are hosted on outside servers. Access to the service is limited to those within an IP range or with a functioning login. For policy or technical reasons, these markets may not be able to talk to firmware behind the firewall.
2. If Inkling (or other firms) offered APIs, would clients use them? Conversely, would clients let Inkling use the APIs of their internal systems? (Ed: Do clients even have APIs for their firmware that can be used to talk with Inkling? Many don’t.)
3. How much connectivity with corporate services is necessary? A one-time dump of HR data into Inkling, or a one-time paste of JavaScript code into a dashboard page would be an easy way to integrate markets. How many of these opportunities exist? How feasible are they?
4. Firmware providers such as Oracle, SalesForce or MicroStrategy could be best positioned to deliver integrated PMs. Will they ever make an investment in developing or acquiring PM software? Would they be receptive to a sales pitch to acquire a startup?
5. Can corporate PMs (and corporate PM companies) flourish without a major firmware provider for distribution?

Going the firmware integration route doesn’t look easy for the current generation of PM startups. Do share your strategies for this (if you don’t consider it too sensitive).

Originally posted at my site.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + Network + Write A Post Or A Page + How To Publish

• Inkling Markets’ growth and success
• Inkling Markets goes local. + Inkling Markets got manipulated. + Inkling Markets cutting down prediction market consultants.
• Adam Siegel of Inkling Markets has computed that blogging is important to a small business.
• The answer from Inkling Markets
• Are Consensus Point and Inkling Markets’ successes spectacular or unspectacular??

[Yahoo! Confab on prediction markets. Here's the transcript of Google's Bo Cowgill's December 13, 2006's presentation.]

—

Thank you for having me. Today I’m going to be talking about how prediction market operators can best encourage participation. I don’t think that rewards — even cash rewards — are adequate to induce participation. Market operators should instead focus on social rewards and the infrastructure and processes to deliver them.

In the classic view of prediction markets, financial self-interest drives participation. In this view, confidentiality is a good thing because it allows unpopular views to be heard. Lastly, the information coming from this market comes entirely in the form of prices and prices only.

I’d like to take issue with all of these points, and then propose solutions. First, it isn’t as easy to drive participation using economic self-interest as one might think. There are several reasons why prizes do not motivate people to participate, even when you’re using cash prizes.

The first reason is complexity and uncertainty. To avoid regulation as gambling, prediction market operators have created complicated payoff mechanisms that are difficult for players to understand. Payoffs often depend not only on whether the user has made correct predictions, but also on how other players bid and how many other players participate at all. This makes it difficult for players to understand how they will be rewarded.

The larger reason is that expected payoffs are low. Suppose a company has a $40K cash reward for the winner among 1000 participants — a bigger reward scheme than anything I’ve seen in the real world. Even with these incentives, I doubt how well this would motivate participation.

Without knowing more about how well you will perform against others, the expected value is about $40. For the markets to work best, traders must be constantly engaged. They must be updating the markets as new information arrives. Is $40 enough to encourage that type of monitoring if you’re a busy, well-paid professional making over $100K annually? I don’t think it is. And in reality, the odds are often worse.

As a side note, I don’t think the same psychological dynamics would apply if people could use their own money. That’s not the world we’re living in, though. So I believe the answer is in rewarding participants with some form of reputation system.

A reputation-based rewards system should involve a prominent leader board for participants in the market – ideally displayed on company’s intranet homepage. Perhaps there can be several different categories of winners. Each employee’s listing in the company directory should include his or her ranking or holdings in the system.

Data from an HR database could show participants which teams, job functions or management levels are performing best in the markets. This would not only help market liquidity, but also foster camaraderie among teams. Management should bestow additional public honors to the best traders as well.

In a large company, a reputation system will encounter problems. It is hard for employees to care about their ranking if it is in the thousands — even if they are ranked relatively high. The market software should allow users to define a group of peers for comparison purposes. Employees might choose friends, co-workers, people who started with them or anyone else they compare themselves with. The software could pre-populate this list with people in the employees’ team, department, starting class or frequent meeting or email contacts.

In addition, the market should provide each player his or her own profile page — complete with picture and introduction — where his or her best trades can be displayed. Confidentiality in prediction markets is overrated. It prevents traders from sharing their insights and using the market as a badge of their insight.

How often inside corporations do you hear some variant of ‘I told you so?’ Even more often, people are thinking it and aren’t saying it. There is demand among employees to take credit for accurate predictions on particular projects. To drive participation, market operators need to give employees the tools the need to share their predictions.

Future generations of corporate prediction markets should make confidentiality optional and disclosure by default. One hybrid way of gaining the benefits of both confidentiality and openness is: Create an option by which trades are confidential until the market closes, at which time money-earning shares are displayed. This would allow traders to share their accomplishments without creating copycat traders while the markets are open. With some imagination, there are other ways that copycat trading could be prevented.

Being able to see who bet how could make the markets more interesting to watch and participate in. In addition to sharing the financial dimensions of trades, the market should allow traders to append an explanation. This, too, should be shared on a voluntary basis. It would show the world that the trader’s fortune was the result of genuine insight and not fortuitous happenstance.

Adding trade explanations would also help make the market more useful from management’s perspective. Without explanations for trades, it is hard for managers to have faith in what the market is saying. To make explaining as easy as possible, traders could be shown a list of reasons that other traders or observers have already submitted. Rather than typing a new explanation, they could select an existing explanation (or set of explanations) to attach to their orders.

I would like to end on a story that I think demonstrates the superiority of a reputation-based rewards system. This year, the World Cup finals happened right on the edge of the Second and Third Quarters of 2006. So that we could keep these markets open a long time, we included the World Cup in the Second Quarter contest. We left these markets open about a week into Q3, and promptly closed them and opened Q3 markets as soon as the World Cup was over.

As a result, people didn’t have much time to see their final rankings. Emails came pouring in asking to see their final ranks and about who would get a T-shirt. Because of other demands on my time, I didn’t get around to identifying the T-shirt winners until later, but in the meantime emails were pouring in. When I finally sent out the winners email, I accidentally forgot to announce who won our seven large cash prizes. Not a single person noticed. In the meantime, the emails continued coming in about when the shirts would be delivered.

A few weeks later, someone finally wrote in. “You said my rank was below 20, but I didn’t get a shirt,” the person said. Upon investigation, I learned that I removed this person from the shirt list because he also won the cash prize. I realized I hadn’t identified the cash prize winners. The only reason I even knew about it was because of someone’s concern about shirts!

Next, I again became caught up in other work and didn’t deliver the cash prizes for almost a month later — and nobody spoke a word in the meantime. The point is that the traders didn’t seem to care about the cash prizes at all. Instead, people wanted to know about reputational prizes — their ranking in the system and the shirts that would identify them as winners.

Thank you very much, and if you have more questions we can talk during the break.

—

Text first published on Bo Cowgill’s blog.

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + Network + Write A Post Or A Page + How To Publish

• How the Hollywood Stock Exchange and the other betting exchanges encourage event derivative traders to continually check back and trade further
• The participation problem with internal prediction markets, and how e-mail could help.
• A letter to the CFTC about for-profit prediction market exchanges
• How long will the writers’ strike last? And can prediction markets help answer this question?
• Prediction Markets = Clear Expiry + Disperse Information + Participation Incentives

by Google’s Bo Cowgill (Project Manager):

—

At Google, we’re constantly trying to find new ways to organize the world’s information, including information relevant to our business. Building on the ideas of Friedrich Hayek and the Iowa Electronic Markets, a few Googlers (Doug Banks, Patri Friedman, Ilya Kirnos, Piaw Na and me, with some help from Hal Varian), set up a [prediction] market system inside the company.

The markets were designed to forecast product launch dates, new office openings, and many other things of strategic importance to Google. So far, more than a thousand Googlers have bid on 146 events in 43 different subject areas (no payment is required to play).

We designed the market so that the price of an event should, in theory, reflect a consensus probability that the event will occur. To determine accuracy of the market, we looked at the connection between prices of events and the frequency with which they actually occurred. If prices are correct, events priced at 10 cents should occur about 10 percent of the time.

In the graph below, the X-axis indicates the price ranges for the group. The orange line represents the average price, which is how often outcomes in that group should actually happen according to market prices. The purple line is how often they did happen. Ideally these would be equal, and as you can see they’re pretty close. So our prices really do represent probabilities - very exciting!

We also found that the market prices gave decisive, informative predictions in the sense that their predictive power increased as time passed and uncertainty was resolved. When a market first opens there may be considerable uncertainty about what will eventually happen; but as time goes on, some outcomes became more likely than others. The market prices should reflect this phenomenon, with the implied probability distributions becoming more concentrated over time.

Being geeks, we naturally used information theory to measure the entropy of our probability distributions:

In this graph, we have weeks before market expiration on the X-axis, and entropy (in bits) on the Y-axis. We’ve included some reference entropies to help your intuition, and you can see that in addition to accurate predictions, the distributions become steadily more informative and decisive (lower entropy) over time.

Our search engine works well because it aggregates information dispersed across the web, and our internal [prediction] markets are based on the same principle: Googlers from across the company contribute knowledge and opinions which are aggregated into a forecast by the market. Sometimes, just feeling lucky isn’t enough, and these tools can help.

—

Putting crowd wisdom to work - by Bo Cowgill - 2005-09-21 - Published originally on the Official Google Blog

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + Network + Write A Post Or A Page + How To Publish

• James Surowiecki’s wisdom on THE WISDOM OF CROWDS
• The Rise of Crowdsourcing: Creative Wisdom of the Crowd — Tuesday, May 20th, 2008 at 6:00pm — @ Bo Cowgill’s Alma Mater
• Prediction markets fundamentally work on the wisdom of crowds.
• Do Google’s enterprise prediction markets work? — Using Prediction Markets to Track Information Flows: Evidence from Google
• Prediction markets are the best way to tap into the wisdom of crowds.

In its October 25, 2006 press release, Cambridge-based consultancy Market Platform Dynamics says it has the “only China-based prediction market capability.” They actually seem to be referring to a survey. Their website gives no evidence of a price mechanism.

MPD’s partnerships in China give us exclusive access to a network of more than 11,000 Chinese managers who will serve as the firm’s Chinese “prediction market” and who will routinely surveyed on behalf of clients to determine their interests in new products and services.

Anyone know what’s up?

----------

Midas Oracle .ORG = Site Map + Archives + Best + Charts + Exchanges + Software + Links + Network + Write A Post Or A Page + How To Publish

• A Chinese soldier has died of the H5N1 bird-flu strain, the country’s 16th reported death from the virus.
• Google’s policy in China is sound.
• CFTC Requests Public Input on Possible Regulation of “Event Contracts” —a.k.a. event derivative markets, event futures markets, betting markets, bet markets, prediction markets
• At Yahoo! Research, Yiling Chen is David Pennock’s right arm on prediction markets.
• EPS Prediction Markets = Earnings Per Share Prediction Markets — Google