VIDEO – Bo Cowgill on Googles enterprise prediction markets – OReilly Money:Tech

No Gravatar

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

Blip.TV

I already published this video. The reason I do it again is that I found out a hidden function in WordPress to increase the dimensions of the embedded video player. I think it is useful in this particular case because Bo shows us some slides, in this video. So, my hope is that those slides will be more readable that way. Let&#8217-s try that. I am pressing &#8220-publish&#8221-&#8230- let&#8217-s see.

Our previous blog post on the Google paper

Previous blog posts by Chris F. Masse:

  • The CFTC is going to close the comments in 9 days. We have 9 days left to convince the CFTC to accept FOR-PROFIT prediction exchanges (e.g., InTrade USA or BetFair USA), and counter the puritan and sterile petition organized by the American Enterprise Institute (which has on its payroll Paul Wolfowitz, the bright masterminder of the Iraq war).
  • Forrest Nelson valids Emile Servan-Schreiber.
  • Averaging One’s Guesses
  • Americans love rankings, but Americans hate to be assessed subjectively.
  • A libertarian view on the Internet betting and gambling industry in the United States of America
  • The CFTC is going to close the comments in 10 days. We have 10 days left to convince the CFTC to accept FOR-PROFIT prediction exchanges (e.g., InTrade USA or BetFair USA), and counter the puritan and sterile petition organized by the American Enterprise Institute (which has on its payroll Paul Wolfowitz, the bright masterminder of the Iraq war).
  • The Numbers Guy

Using Prediction Markets to Track Information Flows: Evidence from Google – VIDEO – Bo Cowgill on Googles enterprise prediction markets – OReilly Money:Tech

No Gravatar

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

Via Daniel Horowitz (Business and Technology Consultant)

Blip.TV &#8212- (FLV file)

It&#8217-s cool. :-D

Google Web Search

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

ABSTRACT: In the last 2.5 years, Google has conducted the largest corporate experiment with prediction markets we are aware of. In this paper, we illustrate how markets can be used to study how an organization processes information. We document a number of biases in Google’s markets, most notably an optimistic bias. Newly hired employees are on the optimistic side of these markets, and optimistic biases are significantly more pronounced on days when Google stock is appreciating. We find strong correlations in trading for those who sit within a few feet of one another- social networks and work relationships also play a secondary explanatory role. The results are interesting in light of recent research on the role of optimism in entrepreneurial firms, as well as recent work on the importance of geographical and social proximity in explaining information flows in firms and markets.

DISCUSSION: In the past few years, many companies have experimented with prediction markets. In this paper, we analyze the largest such experiment we are aware of. We find that prices in Google’s markets closely approximated event probabilities, but did contain some biases, especially early in our sample. The most interesting of these was an optimism bias, which was more pronounced for subjects under the control of Google employees, such as would a project be completed on time or would a particular office be opened. Optimism was more present in the trading of newly hired employees, and was significantly more pronounced on and immediately following days with Google stock price appreciation. Our optimism results are interesting given the role that optimism is often thought to play in motivation and the success of entrepreneurial firms. They raise the possibility of a “stock price-optimism-performance-stock price” feedback that may be worthy of further investigation. We also examine how information and beliefs about prediction market topics move around an organization. We find a significant role for micro-geography. The trading of physically proximate employees is correlated, and only becomes correlated after the employees begin to sit near each other, suggesting a causal relationship. Work and social connections play a detectable but significantly smaller role.

An important caveat to our results is that they tell us about information flows about prediction market subjects, many of which are ancillary to employees’ main jobs. This may explain why physical proximity matters so much more than work relationships – if prediction market topics are lower-priority subjects on which to exchange information, then information exchange may require the opportunities for low-opportunity-cost communication created by physical proximity. Of course, introspection suggests that genuinely creative ideas often arise from such low-opportunity-cost communication. Google’s frequent office moves and emphasis on product innovation may provide an ideal testing ground in which to better understand the creative process.

PAPER BODY: In the last 4 years, many large firms have begun experimenting with internal prediction markets run among their employees. The primary goal of these markets is to generate predictions that efficiently aggregate many employees’ information and augment existing forecasting methods. […] In this paper, we argue that in addition to making predictions, internal prediction can provide insight into how organizations process information. Prediction markets provide employees with incentives for truthful revelation and can capture changes in opinion at a much higher frequency than surveys, allowing one to track how information moves around an organization and how it responds to external events. […]

We can draw two main conclusions. The first is that Google’s markets, while reasonably efficient, reveal some biases. During our study period, the internal markets overpriced securities tied to optimistic outcomes by 10 percentage points. The optimistic bias in Google’s markets was significantly greater on and following days when Google stock appreciated. Securities tied to extreme outcomes were underpriced by a smaller magnitude, and favorites were also overpriced slightly. These biases in prices were partly driven by the trading of newly hired employees- Google employees with longer tenure and more experience trading in the markets were better calibrated. Perhaps as a result, the pricing biases in Google’s markets declined over our sample period, suggesting that corporate prediction markets may perform better as collective experience increases.

The second conclusion is that opinions on specific topics are correlated among employees who are proximate in some sense. Physical proximity was the most important of the forms of proximity we studied. Physical proximity needed to be extremely close for it to matter. Using data on the precise latitude and longitude of employees’ offices, we found that prediction market positions were most correlated among employees sharing an office, that correlations declined with distance for employees on the same floor of a building, and that employees on different floors of the same building were no more correlated than employees in different cities.4 Google employees moved offices extremely frequently during our sample period (in the US, approximately once every 90 days), and we are able to use these office moves to show that our results are not simply the result of like-minded individuals being seated together. […]

Our findings contribute to three quite different literatures: on the role of optimism in entrepreneurial firms, on employee communication in organizations, and on social networks and information flows among investors. […]

The lessons of the literature informed Google CEO Eric Schmidt and Chief Economist Hal Varian’s (2005) third rule for managing knowledge workers: “Pack Them In.” Indeed, the fact that Google employees moved so frequently during our sample period suggests that considerable thought is put into optimizing physical locations. To this literature, which has largely relied on retrospective surveys to track communication, we illustrate how prediction markets can be used as high-frequency, market-incentivized surveys to track information flows in real-time. […]

Google’s prediction markets were launched in April 2005. The [Google prediction] markets are patterned on the Iowa Electronic Markets (Berg, et. al., 2001). In Google’s terminology, a market asks a question (e.g., “how many users will Gmail have?”) that has 2-5 possible mutually exclusive and completely exhaustive answers (e.g., “Fewer than X users”, “Between X and Y”, and “More than Y”). Each answer corresponds to a security that is worth a unit of currency (called a “Gooble”) if the answer turns out to be correct (and zero otherwise). Trade is conducted via a continuous double auction in each security. As on the IEM, short selling is not allowed– traders can instead exchange a Gooble for a complete set of securities and then sell the ones they choose. Likewise, they can exchange complete set of securities for currency. There is no automated market maker, but several employees did create robotic traders that sometimes played this role.

Each calendar quarter from 2005Q2 to 2007Q3 about 25-30 different markets were created. Participants received a fresh endowment of Goobles which they could invest in securities. The markets’ questions were designed so that they could all be resolved by the end of the quarter. At the end of the quarter, Goobles were converted into raffle tickets and prizes were raffled off. The prize budget was $10,000 per quarter, or about $25-100 per active trader (depending on the number active in a particular quarter). Participation was open to active employees and some contractors and vendors– out of 6,425 employees who had a prediction market account, 1,463 placed at least one trade. […]

Common types of markets included those forecasting demand (e.g., the number of users for a product) and internal performance (e.g., a product’s quality rating, whether a product would leave beta on time). […]

In addition, about 30 percent of Google’s markets were so-called “fun” markets –markets on subjects of interest to its employees but with no clear connection to its business (e.g., the quality of Star Wars Episode III, gas prices, the federal funds rate). Other firms experimenting with prediction markets that we are aware of have avoided these markets, perhaps out of fear of appearing unserious. Interestingly, we find that volume in “fun” and “serious” markets are positively correlated (at the daily, weekly, and monthly frequencies), suggesting that the former might help create, rather than crowd out, liquidity for the latter. […]

Google’s prediction markets are reasonably efficient, but did exhibit four specific biases: an overpricing of favorites, short aversion, optimism, and an underpricing of extreme outcomes. New employees and inexperienced traders appear to suffer more from these biases, and as market participants gained experience over the course of our sample period, the biases become less pronounced. […]

FOOT NOTE: One trader in Google’s markets wrote a trading robot that was extremely prolific and ended up participating in about half of all trades. Many of these trades exploited arbitrage opportunities available from simultaneously selling all securities in a bundle. In order to avoid having this trader dominate the (trade-weighted) results in Table 9, we include a dummy variable to control for him or her. None of the results discussed in the above paragraph are sensitive to removing this dummy variable.

APPENDIX:

Google Chart 1

Google Chart 2

Bo Cowgill&#8217-s precision point on micro-geography:

Below you can see a snapshot of trading in one of our offices. The areas where employees are making profitable decisions is green, and the areas where employees are making unprofitable decisions is red. There are about 16 profitable traders in that big green blotch in the middle!

Chart of the Day: Information Sharing at Google

More information from our previous blog post on the Google paper

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

Prediction Markets at Google – by Peter A. Coles, Karim R. Lakhani, Andrew McAfee

No GravatarAlas, that paper is not free to access.

Andrew McAfee&#8217-s post reveals this:

Prediction markets were (sic) very much like stock markets. They contained securities, each of which had a price. [&#8230-]

Not sure why they used the past tense.

Prediction markets are in fact event derivative markets.

Papers from Robin Hanson, Justin Wolfers, Eric Zitzewitz, Koleman Strumpf, etc., are free to download.

Via George Tziralis, of Ask Markets.

Previously: Do Google’s enterprise prediction markets work?

Previous blog posts by Chris F. Masse:

  • “Annette 15”, the once-hot female poker star sponsored by BetFair Poker, does blog only twice a month on the official BetFair blog… when she blogs at all… if you call that blogging.
  • Inkling Markets bring in awards, honors, advisors, and new clients —leaving competition in the dust.
  • No need of enterprise prediction markets to boost intra-corporation communication
  • Inkling Markets is included in the 2008 list of “Cool Vendors” by Gartner.
  • BetFair-TradeFair has won its second Queen’s Award for Enterprise in its eight-year history.
  • Inkling Markets is one of the “Hot Companies To Watch In 2008”, according to Forrester.
  • Plenty of great news coming from Inkling Markets in the coming weeks

The New York Times article doesnt mention Googles enterprise prediction markets, alas. – Bo Cowgill says that the illustration published in the sidebar defines exclusively what is done at Google.

No Gravatar

Right-click on the New York Times graphic below, open Bo Cowgill&#8217-s post in another browser tab, and read his arguments.

NYT PMs

Image Credit: Chris Gash for the New York Times

Adam Siegel of Inkling Markets is also out with a post on that NYT article, but it is of no intellectual interest. Maybe Adam should blog less quickly and eat more fish.

I forgot to tell you, the other day, that Best Buy is a Consensus Point client, but you knew that already.

Previously: The New York Times is telling the business world that enterprise prediction markets are an essential management tool.

[Via Xpree]

Previously: Do Google’s enterprise prediction markets work?

Previous blog posts by Chris F. Masse:

  • The Terror Finance Blog
  • Playing fantasy sports is not gambling. The Unlawful Internet Gambling Enforcement Act includes a specific exemption for fantasy sports, provided the prizes are determined in advance and the imaginary teams don’t correspond to any real teams.
  • Inkling Markets’ Advisory Board… which does not want to tell its name
  • BetFair created the world’s largest ad banner —as certified by the Guinness Book of World Records.
  • Why Emile Servan-Schreiber is on to something with Bet 2 Give —and why InTrade, TradeSports and BetFair should each have a philanthropy wallet.
  • The CFTC is going to close the comments in 14 days. We have 14 days left to convince the CFTC to accept FOR-PROFIT prediction exchanges, and counter the evil petition organized by the American Enterprise Institute (which has on its payroll Paul Wolfowitz, the bright masterminder of the Iraq war).
  • The purpose of X2 is to identify future disruptions, opportunities, and competitive landscapes related to the content and dynamics of global science and technology innovation- to develop a new platform for understanding global innovation trends- and to present this information to policy- and decision-makers, as well as the general public, in a useful form.

Googles Bo Cowgill takes a swipe at the prediction market software vendors.

No Gravatar

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

Bo Cowgill:

[&#8230-] Trade-by-trade data can reveal characteristics of specific working groups: What they know, how they feel, how they process and share information and how all of that changes over time. I didn&#8217-t try to put any of this in the paper because the conclusions would be sensitive, and I thought this application was pretty obvious to anybody who understood our methodology. [&#8230-]

Bo Cowgill:

I&#8217-ve also heard that other companies would find it impossible to analyze the interaction between their market and the organization. Why? Lack of data. [&#8230-]

Bo Cowgill:

Some more remarks about applications that combine prediction markets and organizational data (org charts, social networks, seating locations). The obstacle to these applications is not a lack of data. Jed mentions privacy concerns &#8212- and if he thinks this is a big obstacle then I&#8217-d be interested in discussing his thoughts.

A bigger problem is that that current prediction market vendors and consultants cannot support these applications. At heart, these vendors are software engineers and salespeople at heart, not statisticians or data miners. They want to write one system that can support lots of clients. At conferences, one hears PM vendors complain about having to do &#8220-customization&#8221- work for clients.

This approach would not work for the applications I describe for two reasons:

  1. The inputs for different clients won&#8217-t be the same. Each client&#8217-s organizational data will likely take a different structure. This makes it difficult for prediction market vendors to architect a single system that can served many clients (yet another challenge with integrating markets with other corporate IT services).
  2. The outputs for different clients won&#8217-t be the same. The business relevance and statistical power of each analysis will differ with each client&#8217-s data.

Prediction market vendors may also need to familiarize themselves with the statistical learning methods necessary to fully utilize these rich datasets. So what&#8217-s the solution? First, move to a software-and-consulting model. By &#8216-consulting,&#8217- I don&#8217-t mean &#8216-consulting on how to implement the market.&#8217- I&#8217-m talking about helping the client solve its problem using a variety of data, including prediction market data.

Second, the vendors also need to pitch prediction markets as more than a forecasting tool. People in the business world commonly identify as data junkies &#8212- probably more so than they identify with the &#8216-wisdom of crowds&#8217- ethos. It is unclear how much companies really care about accurate forecasting anyway.

On a related note, there is something that only the prediction market software vendors could do, at this time, for those who are in capacity to do so: setting up inter-industry prediction markets &#8212-or at least, handing over (with everybody&#8217-s agreement) anonymized prediction market data on industry topics to anyone else in the industry who is a client of that PM firm. I don&#8217-t know about NewsFutures or Inkling Markets, but if you look at Consensus Point&#8217-s list of clients, you&#8217-ll see that David Perry&#8217-s firm is strong in the (consumer) electronic industry &#8212-Motorola, Qualcomm, Siemens, Nokia. Use your imagination, or ask David Perry directly, for more&#8230- (I can&#8217-t talk- otherwise, next thing, I&#8217-m a dead blogger.)&#8230-

Previous blog posts by Chris F. Masse:

  • Last year’s best April Fool’s Day Joke had something to do with the Wisdom Of Crowds.
  • Will HedgeStreet USA, the hypothetical InTrade USA, and the hypothetical TradeFair USA, be regulated in the future by a merged SEC+CFTC regulatory structure?
  • WORST THAN ELIOT SPITZER (if it were possible): Formula One boss, Max Mosley, had sado-masochist sex with 5 prostitutes, for 5 hours (!!), reenacting a concentration camp scene (!!) in which he played the role of both Nazi guard and inmate.
  • Is BetFair Poker a booby trap for the gullible novices? Does The Sporting Exchange (the operator of the BetFair brands) help gangs plucking down innocent recreational poker players?? To get an inkling, don’t read The Guardian, seeded by the BetFair spin doctor- read Midas Oracle.
  • The video that the technologically retarded BetFair spin doctor should watch.