CBS.

Jason Ruspini and David Pennock should listen.

We are the world’s #1 social networking group on prediction markets.

-

-

1. How To Join Us

• From within LinkedIn, FaceBook, or Google Reader / Google Mail, send me (Chris Masse) an invite and I’ll accept it.
• You are also invited to join the Prediction Markets group at LinkedIn. We accept everybody (traders, analysts, researchers, consultants, exchange managers, bloggers, etc.).
  
• [As for joining Midas Oracle as a commenter or poster, see this other webpage.]
• I can introduce you to another member of our business network. Just ask me (Chris Masse), and I’ll do.

-

-

2. LinkedIn Network

-

For your information, here is the listing of the co-managers of the Prediction Markets group at LinkedIn:

• Tony Clare of BetFair
• John Delaney of InTrade
• Nigel Eccles of HubDub
• Chris Hibbert of Zocalo
• Chris Masse of Midas Oracle
• David Pennock of Yahoo! Research
• David Perry of Consensus Point
• Emile Servan-Schreiber of NewsFutures
• Adam Siegel of Inkling Markets

-

-

- The 10th seat of this board should be reserved for a yet-to-be-created Institute for Prediction Markets, so that the change of ownership of this group could be operated, at one time in the distant future.

-

Once you have joined our Prediction Markets group at LinkedIn, here’s how to make its logo visible on your profile.

-

-

- After you have joined our Prediction Markets group at LinkedIn, please make its logo visible on your profile. (In your listing of groups you belong to, you should read, near each one, “change visibility”. Click on that.)

- You are also invited to join Chris Masse’s network at LinkedIn.

-

Our Prediction Market People – (from the two networks listed just above)

- Bernd Ankenbrand (Gexid Manager)

- Maurice Balick (NewsFutures CTO)

- Pierluigi Buccioli (Entrepreneur and Betting Exchange Trader; Owner of Bookmakers Review)

- Jason Carver (Entrepreneur and Engineer)

- Yiling Chen (Professor at Harvard University)

- Tony Clare (Head of Strategic Initiatives at Betfair)

- Norris Clark (Vice-President of Sales at NewsFutures)

- Alexander Costakis (Managing Director of Hollywood Stock Exchange)

- Tyler Cowen (Economics Professor at George Mason University)

- Bo Cowgill (Quantitative Marketing Manager at Google)

- Eric Crampton (Senior Lecturer at University of Canterbury)

- Noam Danon (Qmarkets CEO and Founder)

- Pedro Da Cunha (Exago Markets CEO)

- John Delaney (InTrade CEO)

- Juan Manuel Ducler (Founder, BestChartsOnline.com / Trader & Financial Consultant)

- Nigel Eccles (HudDub CEO)

- Leslie Fine (VP of Market Design at Xpree)

- Matthew Fogarty (Prediction Market Consultant – Xpree CEO and Founder)

- Lance Fortnow (Professor of Computer Science at Northwestern University)

- Nick Garner (SEO / PPC / Search Manager at BetFair)

- Cédric Gaspoz (Research and Teaching Assistant at University of Lausanne)

- Michael Giberson (Energy Economist at Texas Tech University)

- Sean Glass (Founder, Chairman, Chief Strategy Officer at Pikum Holdings)

- Andrew Goldberg (Intern at Media Law Resource Center)

- Robin Hanson (Professor of Economics at George Mason University)

- Chris Hibbert (Software Architect, Project Lead at Zocalo)

- Donna Hoffman (Professor and Co-Director, Sloan Center for Internet Retailing)

- Panagiotis Ipeirotis (Assistant Professor at New York University)

- Max Keiser (Entrepreneur, Journalist, Co-Founder of the Hollywood Stock Exchange)

- Alex Kirtland (User Experience Consultant)

- Ken Kittlitz (Software Architect, Foresight Exchange, Consensus Point)

- Greg Knaddison (Developer and Sys Admin for PingVision)

- Nathan Kontny (Inkling Markets CTO)

- Kriss Monaco (Director of New Product Development at International Securities Exchange)

- Dean LeBaron (Independent Investment Management Professional)

- Heidi Levin (Director of Business Development at Inkling Markets)

- Mike Linksvayer (Vice-President of Creative Commons)

- Rory Mackay (PredictionsMarkets.com Co-Founder and Owner)

- Chris. F. Masse (Founder and President of Midas Oracle)

- Tracy Mullen (Penn State Professor of Information Technology)

- Jesper Muller-Krogstrup (Managing Director of Nosco)

- Sean Park (Founding Partner at Sixth Paradigm)

- David Pennock (Principal Research Scientist at Yahoo!)

- David Perry (Consensus Point President and Co-Founder)

- Daniel Reeves (Yahoo! Research Scientist)

- Steve Roman (Financial Analyst at FXCM)

- Jason Ruspini (Financial Research Analyst, Vice President at Conquest Capital Group)

- Mike Sankowski (Product Development and Market Operations Manager at US Futures Exchange)

- Emile Servan-Schreiber (NewsFutures CEO and Co-Founder)

- Brian Shiau (The Sim Exchange CEO and Founder)

- Adam Siegel (Inkling Markets CEO and Co-Founder)

- Ashish Singal (Capital Markets Professional)

- Erik Snowberg (Assistant Professor of Economics and Political Science at California Institute of Technology)

- Brad Stewart (Reality Markets Founder)

- Brent Stinski (Media Predict CEO and Founder)

- Karim Tahawi (MyCurrency Founder)

- Jason Trost (Co-Founder of Smarkets)

- George Tziralis (Doctoral Researcher at National Technical University of Athens, Co-Founder of AskMarkets)

- Robert Wilburn (Rimdex CEO and Founder)

- Gerry Wilson (YooNew CEO and Co-Founder)

- Justin Wolfers (Professor of Business and Public Policy at the University of Pennsylvania)

- Matt Youill (Chief Technologist at Betfair)

- David Yu (BetFair CEO, Former CTO and COO of BetFair)

- Eric Zitzewitz (Professor of Economics at Dartmouth College)

- Plus, many, many more…

-

Other Prediction Market People At LinkedIn

- Russell Andersson (Chief Operating Officer, Third Ave Beach)

- Paul Architzel (Counsel at Alston & Bird)

- Adrian Asher (Global Head of Security at BetFair)

- Henry Berg (Group Manager on Information Markets at Microsoft)

- Matt Carter (Director of the Advanced Technology Group at BetFair)

- Jonathan Cumberlege (Former Director of Registrations & Payments at BetFair)

- Gerard Cunningham (President at BetFair USA)

- Mark Davies (Managing Director of Corporate Affairs of BetFair)

- Mike Dooley (Vice-President of Engineering at NewsFutures)

- Sean Dunbar (Former Head of Technology at Hollywood Stock Exchange)

- Mathias Entenmann (Exchange Managing Director of BetFair)

- Brian Galebach (Freelance Computer Programmer and Owner of Probability Sports)

- Carol Gebert (Former Founder at Incentive Markets)

- Christian Hellmers (Director of US Business Development of BetFair)

- David Jack (TradeFair Director)

- Nicholas Jenkins (Owner of Betcha.com)

- Ajit Kambil (Global Director at Deloitte Research)

- Alam Kasenally (Xpree CTO)

- Dawn Tevekelian Keller (Director, Services Business Group & Prediction Markets at Best Buy)

- Robin Marks (Head of Media at BetFair)

- Hunter Morris (Co-Founder of Smarkets)

- George Neumann (Professor of Economics And Applied Mathematics and Computational Sciences at the University of Iowa, Co-Founder of the Iowa Electronic Markets)

- Paul Pluschkell (Founder & CEO at Spigit)

- Todd Proebsting (Director at Microsoft, Former Group Manager on Information Markets at Microsoft)

- Don Reynolds (Site Administrator at NewsFutures)

- Felix Salmon (Financial Journalist at Portfolio)

- Bimal Shah (Product Manager at TradeFair)

- Martin Spann (Professor at the University of Passau)

- Will Speck (Director Business Development & Market Research at Financial Times & FT.com)

- Martin Thompson (Engineering Director at TradeFair)

- Geoffrey Tso (Engineering Manager at Xpree)

- Andrew Twaits (Corporate and Business Affairs Director of BetFair Australia)

- Mark Wood (Head of Programme at TradeFair0

- Plus, many, many more…

-

We are the world’s #1 social networking group on prediction markets.

-

-

1. How To Join Us

• From within LinkedIn, FaceBook, or Google Reader / Google Mail, send me (Chris Masse) an invite and I’ll accept it.
• You are also invited to join the Prediction Markets group at LinkedIn. We accept everybody (traders, analysts, researchers, consultants, exchange managers, bloggers, etc.).
  
• [As for joining Midas Oracle as a commenter or poster, see this other webpage.]
• I can introduce you to another member of our business network. Just ask me (Chris Masse), and I’ll do.

-

-

2. LinkedIn Network

-

For your information, here is the listing of the co-managers of the Prediction Markets group at LinkedIn:

• Tony Clare of BetFair
• John Delaney of InTrade
• Nigel Eccles of HubDub
• Chris Hibbert of Zocalo
• Chris Masse of Midas Oracle
• David Pennock of Yahoo! Research
• David Perry of Consensus Point
• Emile Servan-Schreiber of NewsFutures
• Adam Siegel of Inkling Markets

-

-

- The 10th seat of this board should be reserved for a yet-to-be-created Institute for Prediction Markets, so that the change of ownership of this group could be operated, at one time in the distant future.

-

Once you have joined our Prediction Markets group at LinkedIn, here’s how to make its logo visible on your profile.

-

-

- After you have joined our Prediction Markets group at LinkedIn, please make its logo visible on your profile. (In your listing of groups you belong to, you should read, near each one, “change visibility”. Click on that.)

- You are also invited to join Chris Masse’s network at LinkedIn.

-

Our Prediction Market People – (from the two networks listed just above)

- Bernd Ankenbrand (Gexid Manager)

- Maurice Balick (NewsFutures CTO)

- Pierluigi Buccioli (Entrepreneur and Betting Exchange Trader; Owner of Bookmakers Review)

- Jason Carver (Entrepreneur and Engineer)

- Yiling Chen (Professor at Harvard University)

- Tony Clare (Head of Strategic Initiatives at Betfair)

- Norris Clark (Vice-President of Sales at NewsFutures)

- Alexander Costakis (Managing Director of Hollywood Stock Exchange)

- Tyler Cowen (Economics Professor at George Mason University)

- Bo Cowgill (Quantitative Marketing Manager at Google)

- Eric Crampton (Senior Lecturer at University of Canterbury)

- Noam Danon (Qmarkets CEO and Founder)

- Pedro Da Cunha (Exago Markets CEO)

- John Delaney (InTrade CEO)

- Juan Manuel Ducler (Founder, BestChartsOnline.com / Trader & Financial Consultant)

- Nigel Eccles (HudDub CEO)

- Leslie Fine (VP of Market Design at Xpree)

- Matthew Fogarty (Prediction Market Consultant – Xpree CEO and Founder)

- Lance Fortnow (Professor of Computer Science at Northwestern University)

- Nick Garner (SEO / PPC / Search Manager at BetFair)

- Cédric Gaspoz (Research and Teaching Assistant at University of Lausanne)

- Michael Giberson (Energy Economist at Texas Tech University)

- Sean Glass (Founder, Chairman, Chief Strategy Officer at Pikum Holdings)

- Andrew Goldberg (Intern at Media Law Resource Center)

- Robin Hanson (Professor of Economics at George Mason University)

- Chris Hibbert (Software Architect, Project Lead at Zocalo)

- Donna Hoffman (Professor and Co-Director, Sloan Center for Internet Retailing)

- Panagiotis Ipeirotis (Assistant Professor at New York University)

- Max Keiser (Entrepreneur, Journalist, Co-Founder of the Hollywood Stock Exchange)

- Alex Kirtland (User Experience Consultant)

- Ken Kittlitz (Software Architect, Foresight Exchange, Consensus Point)

- Greg Knaddison (Developer and Sys Admin for PingVision)

- Nathan Kontny (Inkling Markets CTO)

- Kriss Monaco (Director of New Product Development at International Securities Exchange)

- Dean LeBaron (Independent Investment Management Professional)

- Heidi Levin (Director of Business Development at Inkling Markets)

- Mike Linksvayer (Vice-President of Creative Commons)

- Rory Mackay (PredictionsMarkets.com Co-Founder and Owner)

- Chris. F. Masse (Founder and President of Midas Oracle)

- Tracy Mullen (Penn State Professor of Information Technology)

- Jesper Muller-Krogstrup (Managing Director of Nosco)

- Sean Park (Founding Partner at Sixth Paradigm)

- David Pennock (Principal Research Scientist at Yahoo!)

- David Perry (Consensus Point President and Co-Founder)

- Daniel Reeves (Yahoo! Research Scientist)

- Steve Roman (Financial Analyst at FXCM)

- Jason Ruspini (Financial Research Analyst, Vice President at Conquest Capital Group)

- Mike Sankowski (Product Development and Market Operations Manager at US Futures Exchange)

- Emile Servan-Schreiber (NewsFutures CEO and Co-Founder)

- Brian Shiau (The Sim Exchange CEO and Founder)

- Adam Siegel (Inkling Markets CEO and Co-Founder)

- Ashish Singal (Capital Markets Professional)

- Erik Snowberg (Assistant Professor of Economics and Political Science at California Institute of Technology)

- Brad Stewart (Reality Markets Founder)

- Brent Stinski (Media Predict CEO and Founder)

- Karim Tahawi (MyCurrency Founder)

- Jason Trost (Co-Founder of Smarkets)

- George Tziralis (Doctoral Researcher at National Technical University of Athens, Co-Founder of AskMarkets)

- Robert Wilburn (Rimdex CEO and Founder)

- Gerry Wilson (YooNew CEO and Co-Founder)

- Justin Wolfers (Professor of Business and Public Policy at the University of Pennsylvania)

- Matt Youill (Chief Technologist at Betfair)

- David Yu (BetFair CEO, Former CTO and COO of BetFair)

- Eric Zitzewitz (Professor of Economics at Dartmouth College)

- Plus, many, many more…

-

Other Prediction Market People At LinkedIn

- Russell Andersson (Chief Operating Officer, Third Ave Beach)

- Paul Architzel (Counsel at Alston & Bird)

- Adrian Asher (Global Head of Security at BetFair)

- Henry Berg (Group Manager on Information Markets at Microsoft)

- Matt Carter (Director of the Advanced Technology Group at BetFair)

- Jonathan Cumberlege (Former Director of Registrations & Payments at BetFair)

- Gerard Cunningham (President at BetFair USA)

- Mark Davies (Managing Director of Corporate Affairs of BetFair)

- Mike Dooley (Vice-President of Engineering at NewsFutures)

- Sean Dunbar (Former Head of Technology at Hollywood Stock Exchange)

- Mathias Entenmann (Exchange Managing Director of BetFair)

- Brian Galebach (Freelance Computer Programmer and Owner of Probability Sports)

- Carol Gebert (Former Founder at Incentive Markets)

- Christian Hellmers (Director of US Business Development of BetFair)

- David Jack (TradeFair Director)

- Nicholas Jenkins (Owner of Betcha.com)

- Ajit Kambil (Global Director at Deloitte Research)

- Alam Kasenally (Xpree CTO)

- Dawn Tevekelian Keller (Director, Services Business Group & Prediction Markets at Best Buy)

- Robin Marks (Head of Media at BetFair)

- Hunter Morris (Co-Founder of Smarkets)

- George Neumann (Professor of Economics And Applied Mathematics and Computational Sciences at the University of Iowa, Co-Founder of the Iowa Electronic Markets)

- Paul Pluschkell (Founder & CEO at Spigit)

- Todd Proebsting (Director at Microsoft, Former Group Manager on Information Markets at Microsoft)

- Don Reynolds (Site Administrator at NewsFutures)

- Felix Salmon (Financial Journalist at Portfolio)

- Bimal Shah (Product Manager at TradeFair)

- Martin Spann (Professor at the University of Passau)

- Will Speck (Director Business Development & Market Research at Financial Times & FT.com)

- Martin Thompson (Engineering Director at TradeFair)

- Geoffrey Tso (Engineering Manager at Xpree)

- Andrew Twaits (Corporate and Business Affairs Director of BetFair Australia)

- Mark Wood (Head of Programme at TradeFair0

- Plus, many, many more…

-

Gartner:

1. Multicore and hybrid processors
2. Virtualization and fabric computing
3. Social networks and social software
4. Cloud computing and cloud/Web platforms
5. Web mashups
6. User Interface
7. Ubiquitous computing
8. Contextual computing
9. Augmented reality
10. Semantics

-

Via Daniel Horowitz (Business and Technology Consultant)

Blip.TV — (FLV file)

-

-

-

It’s cool.

-

-

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

-

ABSTRACT: In the last 2.5 years, Google has conducted the largest corporate experiment with prediction markets we are aware of. In this paper, we illustrate how markets can be used to study how an organization processes information. We document a number of biases in Google’s markets, most notably an optimistic bias. Newly hired employees are on the optimistic side of these markets, and optimistic biases are significantly more pronounced on days when Google stock is appreciating. We find strong correlations in trading for those who sit within a few feet of one another; social networks and work relationships also play a secondary explanatory role. The results are interesting in light of recent research on the role of optimism in entrepreneurial firms, as well as recent work on the importance of geographical and social proximity in explaining information flows in firms and markets.

-

DISCUSSION: In the past few years, many companies have experimented with prediction markets. In this paper, we analyze the largest such experiment we are aware of. We find that prices in Google’s markets closely approximated event probabilities, but did contain some biases, especially early in our sample. The most interesting of these was an optimism bias, which was more pronounced for subjects under the control of Google employees, such as would a project be completed on time or would a particular office be opened. Optimism was more present in the trading of newly hired employees, and was significantly more pronounced on and immediately following days with Google stock price appreciation. Our optimism results are interesting given the role that optimism is often thought to play in motivation and the success of entrepreneurial firms. They raise the possibility of a “stock price-optimism-performance-stock price” feedback that may be worthy of further investigation. We also examine how information and beliefs about prediction market topics move around an organization. We find a significant role for micro-geography. The trading of physically proximate employees is correlated, and only becomes correlated after the employees begin to sit near each other, suggesting a causal relationship. Work and social connections play a detectable but significantly smaller role.

An important caveat to our results is that they tell us about information flows about prediction market subjects, many of which are ancillary to employees’ main jobs. This may explain why physical proximity matters so much more than work relationships – if prediction market topics are lower-priority subjects on which to exchange information, then information exchange may require the opportunities for low-opportunity-cost communication created by physical proximity. Of course, introspection suggests that genuinely creative ideas often arise from such low-opportunity-cost communication. Google’s frequent office moves and emphasis on product innovation may provide an ideal testing ground in which to better understand the creative process.

-

PAPER BODY: In the last 4 years, many large firms have begun experimenting with internal prediction markets run among their employees. The primary goal of these markets is to generate predictions that efficiently aggregate many employees’ information and augment existing forecasting methods. [...] In this paper, we argue that in addition to making predictions, internal prediction can provide insight into how organizations process information. Prediction markets provide employees with incentives for truthful revelation and can capture changes in opinion at a much higher frequency than surveys, allowing one to track how information moves around an organization and how it responds to external events. [...]

We can draw two main conclusions. The first is that Google’s markets, while reasonably efficient, reveal some biases. During our study period, the internal markets overpriced securities tied to optimistic outcomes by 10 percentage points. The optimistic bias in Google’s markets was significantly greater on and following days when Google stock appreciated. Securities tied to extreme outcomes were underpriced by a smaller magnitude, and favorites were also overpriced slightly. These biases in prices were partly driven by the trading of newly hired employees; Google employees with longer tenure and more experience trading in the markets were better calibrated. Perhaps as a result, the pricing biases in Google’s markets declined over our sample period, suggesting that corporate prediction markets may perform better as collective experience increases.

The second conclusion is that opinions on specific topics are correlated among employees who are proximate in some sense. Physical proximity was the most important of the forms of proximity we studied. Physical proximity needed to be extremely close for it to matter. Using data on the precise latitude and longitude of employees’ offices, we found that prediction market positions were most correlated among employees sharing an office, that correlations declined with distance for employees on the same floor of a building, and that employees on different floors of the same building were no more correlated than employees in different cities.4 Google employees moved offices extremely frequently during our sample period (in the US, approximately once every 90 days), and we are able to use these office moves to show that our results are not simply the result of like-minded individuals being seated together. [...]

Our findings contribute to three quite different literatures: on the role of optimism in entrepreneurial firms, on employee communication in organizations, and on social networks and information flows among investors. [...]

The lessons of the literature informed Google CEO Eric Schmidt and Chief Economist Hal Varian’s (2005) third rule for managing knowledge workers: “Pack Them In.” Indeed, the fact that Google employees moved so frequently during our sample period suggests that considerable thought is put into optimizing physical locations. To this literature, which has largely relied on retrospective surveys to track communication, we illustrate how prediction markets can be used as high-frequency, market-incentivized surveys to track information flows in real-time. [...]

Google’s prediction markets were launched in April 2005. The [Google prediction] markets are patterned on the Iowa Electronic Markets (Berg, et. al., 2001). In Google’s terminology, a market asks a question (e.g., “how many users will Gmail have?”) that has 2-5 possible mutually exclusive and completely exhaustive answers (e.g., “Fewer than X users”, “Between X and Y”, and “More than Y”). Each answer corresponds to a security that is worth a unit of currency (called a “Gooble”) if the answer turns out to be correct (and zero otherwise). Trade is conducted via a continuous double auction in each security. As on the IEM, short selling is not allowed; traders can instead exchange a Gooble for a complete set of securities and then sell the ones they choose. Likewise, they can exchange complete set of securities for currency. There is no automated market maker, but several employees did create robotic traders that sometimes played this role.

Each calendar quarter from 2005Q2 to 2007Q3 about 25-30 different markets were created. Participants received a fresh endowment of Goobles which they could invest in securities. The markets’ questions were designed so that they could all be resolved by the end of the quarter. At the end of the quarter, Goobles were converted into raffle tickets and prizes were raffled off. The prize budget was $10,000 per quarter, or about $25-100 per active trader (depending on the number active in a particular quarter). Participation was open to active employees and some contractors and vendors; out of 6,425 employees who had a prediction market account, 1,463 placed at least one trade. [...]

Common types of markets included those forecasting demand (e.g., the number of users for a product) and internal performance (e.g., a product’s quality rating, whether a product would leave beta on time). [...]

In addition, about 30 percent of Google’s markets were so-called “fun” markets –markets on subjects of interest to its employees but with no clear connection to its business (e.g., the quality of Star Wars Episode III, gas prices, the federal funds rate). Other firms experimenting with prediction markets that we are aware of have avoided these markets, perhaps out of fear of appearing unserious. Interestingly, we find that volume in “fun” and “serious” markets are positively correlated (at the daily, weekly, and monthly frequencies), suggesting that the former might help create, rather than crowd out, liquidity for the latter. [...]

Google’s prediction markets are reasonably efficient, but did exhibit four specific biases: an overpricing of favorites, short aversion, optimism, and an underpricing of extreme outcomes. New employees and inexperienced traders appear to suffer more from these biases, and as market participants gained experience over the course of our sample period, the biases become less pronounced. [...]

-

FOOT NOTE: One trader in Google’s markets wrote a trading robot that was extremely prolific and ended up participating in about half of all trades. Many of these trades exploited arbitrage opportunities available from simultaneously selling all securities in a bundle. In order to avoid having this trader dominate the (trade-weighted) results in Table 9, we include a dummy variable to control for him or her. None of the results discussed in the above paragraph are sensitive to removing this dummy variable.

-

APPENDIX:

-

Bo Cowgill’s precision point on micro-geography:

Below you can see a snapshot of trading in one of our offices. The areas where employees are making profitable decisions is green, and the areas where employees are making unprofitable decisions is red. There are about 16 profitable traders in that big green blotch in the middle!

-

More information from our previous blog post on the Google paper

-

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

-

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

-

Bo Cowgill:

[...] Trade-by-trade data can reveal characteristics of specific working groups: What they know, how they feel, how they process and share information and how all of that changes over time. I didn’t try to put any of this in the paper because the conclusions would be sensitive, and I thought this application was pretty obvious to anybody who understood our methodology. [...]

Bo Cowgill:

I’ve also heard that other companies would find it impossible to analyze the interaction between their market and the organization. Why? Lack of data. [...]

Bo Cowgill:

Some more remarks about applications that combine prediction markets and organizational data (org charts, social networks, seating locations). The obstacle to these applications is not a lack of data. Jed mentions privacy concerns — and if he thinks this is a big obstacle then I’d be interested in discussing his thoughts.

A bigger problem is that that current prediction market vendors and consultants cannot support these applications. At heart, these vendors are software engineers and salespeople at heart, not statisticians or data miners. They want to write one system that can support lots of clients. At conferences, one hears PM vendors complain about having to do “customization” work for clients.

This approach would not work for the applications I describe for two reasons:

1. The inputs for different clients won’t be the same. Each client’s organizational data will likely take a different structure. This makes it difficult for prediction market vendors to architect a single system that can served many clients (yet another challenge with integrating markets with other corporate IT services).
2. The outputs for different clients won’t be the same. The business relevance and statistical power of each analysis will differ with each client’s data.

Prediction market vendors may also need to familiarize themselves with the statistical learning methods necessary to fully utilize these rich datasets. So what’s the solution? First, move to a software-and-consulting model. By ‘consulting,’ I don’t mean ‘consulting on how to implement the market.’ I’m talking about helping the client solve its problem using a variety of data, including prediction market data.

Second, the vendors also need to pitch prediction markets as more than a forecasting tool. People in the business world commonly identify as data junkies — probably more so than they identify with the ‘wisdom of crowds’ ethos. It is unclear how much companies really care about accurate forecasting anyway.

-

VIDEO — Bo Cowgill on Google’s enterprise prediction markets — O’Reilly Money:Tech

Blip.TV — (FLV file)

-

-

-

-

I’ll first publish some excerpts from the paper on Google’s enterprise prediction markets (set up by Bo Cowgill, Doug Banks, Patri Friedman, Ilya Kirnos, Piaw Na and Hal Varian), and then I’ll list all the interesting discussion points that this paper has generated on the Web…

-

Here’s Bo Cowgill’s face pic, for those who don’t know him yet:

Bo Cowgill – LinkedIn profile

-

-

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

-

ABSTRACT: In the last 2.5 years, Google has conducted the largest corporate experiment with prediction markets we are aware of. In this paper, we illustrate how markets can be used to study how an organization processes information. We document a number of biases in Google’s markets, most notably an optimistic bias. Newly hired employees are on the optimistic side of these markets, and optimistic biases are significantly more pronounced on days when Google stock is appreciating. We find strong correlations in trading for those who sit within a few feet of one another; social networks and work relationships also play a secondary explanatory role. The results are interesting in light of recent research on the role of optimism in entrepreneurial firms, as well as recent work on the importance of geographical and social proximity in explaining information flows in firms and markets.

-

DISCUSSION: In the past few years, many companies have experimented with prediction markets. In this paper, we analyze the largest such experiment we are aware of. We find that prices in Google’s markets closely approximated event probabilities, but did contain some biases, especially early in our sample. The most interesting of these was an optimism bias, which was more pronounced for subjects under the control of Google employees, such as would a project be completed on time or would a particular office be opened. Optimism was more present in the trading of newly hired employees, and was significantly more pronounced on and immediately following days with Google stock price appreciation. Our optimism results are interesting given the role that optimism is often thought to play in motivation and the success of entrepreneurial firms. They raise the possibility of a “stock price-optimism-performance-stock price” feedback that may be worthy of further investigation. We also examine how information and beliefs about prediction market topics move around an organization. We find a significant role for micro-geography. The trading of physically proximate employees is correlated, and only becomes correlated after the employees begin to sit near each other, suggesting a causal relationship. Work and social connections play a detectable but significantly smaller role.

An important caveat to our results is that they tell us about information flows about prediction market subjects, many of which are ancillary to employees’ main jobs. This may explain why physical proximity matters so much more than work relationships – if prediction market topics are lower-priority subjects on which to exchange information, then information exchange may require the opportunities for low-opportunity-cost communication created by physical proximity. Of course, introspection suggests that genuinely creative ideas often arise from such low-opportunity-cost communication. Google’s frequent office moves and emphasis on product innovation may provide an ideal testing ground in which to better understand the creative process.

-

PAPER BODY: In the last 4 years, many large firms have begun experimenting with internal prediction markets run among their employees. The primary goal of these markets is to generate predictions that efficiently aggregate many employees’ information and augment existing forecasting methods. [...] In this paper, we argue that in addition to making predictions, internal prediction can provide insight into how organizations process information. Prediction markets provide employees with incentives for truthful revelation and can capture changes in opinion at a much higher frequency than surveys, allowing one to track how information moves around an organization and how it responds to external events. [...]

We can draw two main conclusions. The first is that Google’s markets, while reasonably efficient, reveal some biases. During our study period, the internal markets overpriced securities tied to optimistic outcomes by 10 percentage points. The optimistic bias in Google’s markets was significantly greater on and following days when Google stock appreciated. Securities tied to extreme outcomes were underpriced by a smaller magnitude, and favorites were also overpriced slightly. These biases in prices were partly driven by the trading of newly hired employees; Google employees with longer tenure and more experience trading in the markets were better calibrated. Perhaps as a result, the pricing biases in Google’s markets declined over our sample period, suggesting that corporate prediction markets may perform better as collective experience increases.

The second conclusion is that opinions on specific topics are correlated among employees who are proximate in some sense. Physical proximity was the most important of the forms of proximity we studied. Physical proximity needed to be extremely close for it to matter. Using data on the precise latitude and longitude of employees’ offices, we found that prediction market positions were most correlated among employees sharing an office, that correlations declined with distance for employees on the same floor of a building, and that employees on different floors of the same building were no more correlated than employees in different cities.4 Google employees moved offices extremely frequently during our sample period (in the US, approximately once every 90 days), and we are able to use these office moves to show that our results are not simply the result of like-minded individuals being seated together. [...]

Our findings contribute to three quite different literatures: on the role of optimism in entrepreneurial firms, on employee communication in organizations, and on social networks and information flows among investors. [...]

The lessons of the literature informed Google CEO Eric Schmidt and Chief Economist Hal Varian’s (2005) third rule for managing knowledge workers: “Pack Them In.” Indeed, the fact that Google employees moved so frequently during our sample period suggests that considerable thought is put into optimizing physical locations. To this literature, which has largely relied on retrospective surveys to track communication, we illustrate how prediction markets can be used as high-frequency, market-incentivized surveys to track information flows in real-time. [...]

Google’s prediction markets were launched in April 2005. The [Google prediction] markets are patterned on the Iowa Electronic Markets (Berg, et. al., 2001). In Google’s terminology, a market asks a question (e.g., “how many users will Gmail have?”) that has 2-5 possible mutually exclusive and completely exhaustive answers (e.g., “Fewer than X users”, “Between X and Y”, and “More than Y”). Each answer corresponds to a security that is worth a unit of currency (called a “Gooble”) if the answer turns out to be correct (and zero otherwise). Trade is conducted via a continuous double auction in each security. As on the IEM, short selling is not allowed; traders can instead exchange a Gooble for a complete set of securities and then sell the ones they choose. Likewise, they can exchange complete set of securities for currency. There is no automated market maker, but several employees did create robotic traders that sometimes played this role.

Each calendar quarter from 2005Q2 to 2007Q3 about 25-30 different markets were created. Participants received a fresh endowment of Goobles which they could invest in securities. The markets’ questions were designed so that they could all be resolved by the end of the quarter. At the end of the quarter, Goobles were converted into raffle tickets and prizes were raffled off. The prize budget was $10,000 per quarter, or about $25-100 per active trader (depending on the number active in a particular quarter). Participation was open to active employees and some contractors and vendors; out of 6,425 employees who had a prediction market account, 1,463 placed at least one trade. [...]

Common types of markets included those forecasting demand (e.g., the number of users for a product) and internal performance (e.g., a product’s quality rating, whether a product would leave beta on time). [...]

In addition, about 30 percent of Google’s markets were so-called “fun” markets –markets on subjects of interest to its employees but with no clear connection to its business (e.g., the quality of Star Wars Episode III, gas prices, the federal funds rate). Other firms experimenting with prediction markets that we are aware of have avoided these markets, perhaps out of fear of appearing unserious. Interestingly, we find that volume in “fun” and “serious” markets are positively correlated (at the daily, weekly, and monthly frequencies), suggesting that the former might help create, rather than crowd out, liquidity for the latter. [...]

Google’s prediction markets are reasonably efficient, but did exhibit four specific biases: an overpricing of favorites, short aversion, optimism, and an underpricing of extreme outcomes. New employees and inexperienced traders appear to suffer more from these biases, and as market participants gained experience over the course of our sample period, the biases become less pronounced. [...]

-

FOOT NOTE: One trader in Google’s markets wrote a trading robot that was extremely prolific and ended up participating in about half of all trades. Many of these trades exploited arbitrage opportunities available from simultaneously selling all securities in a bundle. In order to avoid having this trader dominate the (trade-weighted) results in Table 9, we include a dummy variable to control for him or her. None of the results discussed in the above paragraph are sensitive to removing this dummy variable.

-

APPENDIX:

-

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

-

Via Zubin Jelveh, Bo Cowgill’s precision point on micro-geography:

Below you can see a snapshot of trading in one of our offices. The areas where employees are making profitable decisions is green, and the areas where employees are making unprofitable decisions is red. There are about 16 profitable traders in that big green blotch in the middle!

-

Eric Zitzewitz in The New York Times:

Peoples’ primary job isn’t to trade these commodities. What we are picking up is communication on ‘low priority topics.’ But that’s how creative ideas come about.

-

Bo Cowgill in Network World:

If you let people bet on things anonymously, they will tell you what they really believe because they have money at stake. This is a conversation that’s happening without politics. Nobody knows who each other is, and nobody has any incentive to kiss up. [...]

As a philosophical matter, Google likes to pack people in tight… so they can share information. As a company gets larger, people don’t always adhere to the founding tenets, or if they’re in a big building they’ll spread out because it’s more comfortable. This was something where we could say ‘there’s value and we can measure that, and we can compare it to what it’s like when people e-mail.’

-

Bo Cowgill (answering Mat Fogarty’s remark):

[...] I don’t think that short-aversion explains the optimistic bias revealed in the Google markets. Because of the IEM-like structure of the markets, no shorting was necessary to make a pessimistic bet. If you wanted to bet pessimistically, you could simply buy the pessimistic outcome. In other words: The optimistic bias was not only saw an aversion to shorting optimistic outcomes. It was also an aversion to buying pessimistic ones. We did observe a general aversion to shorting, which is bad for the market’s efficiency and accuracy. Better terminology could overcome this. More on these two biases in a forthcoming post.

-

The 12 principles that guide programming at Google:

1. All developers work out of a ~single source depot; shared infrastructure!
2. A developer can fix bugs anywhere in the source tree.
3. Building a product takes 3 commands (“get, config, make”)
4. Uniform coding style guidelines across company
5. Code reviews mandatory for all checkins
6. Pervasive unit testing, written by developers
7. Unit tests run continuously, email sent on failure
8. Powerful tools, shared company-wide
9. Rapid project cycles; developers change projects often; 20% time
10. Peer-driven review process; flat management structure
11. Transparency into projects, code, process, ideas, etc.
12. Dozens of offices around world => hire best people regardless of location

The Google paper made a social graph of these code reviews and compared trading habits among circle of co-reviewers. See Table #2 and Table #3.

-

Economist Michael Giberson has more on “Prediction markets and the flow of information inside organizations“.

[...] With private, anonymous trading, the choice to disclose information via the market dramatically revises the mental decision calculus involved. The trader is rewarded if right and penalized if wrong, but in either case the disclosure and net reward is a private matter rather than social event. (At least until you brag about it around the water cooler.) [C]andor should be rewarded and incentives designed to encourage it. Prediction markets provide incentives for candor. Not only that, but over time the traders with useful candor are encouraged by accumulated gains, while blowhards find their accounts diminished. It is true that prediction market prices present relatively limited signals. Prices may go up or down, but they never say why. But with a signal, at least someone knows to start asking “why” and that is better than not knowing.

-

Concurring with Justin Wolfers, here’s Tim Harford on the flow of information around Google’s physical office spaces:

[...] The results were striking. Clear correlations existed between the trading behaviour of certain groups of employees. But they were not explained by shared interests or by social connections. Having the same immediate boss only explains a little about information flows. No, it is the office layout that matters: people who sit near each other tend to know the same things, as evidenced by making similar trades on the prediction markets. Social and professional proximity matters very little for the flow of information: physical proximity is almost everything. Specialists in organisational behaviour have known for a while that people tend to interact much more with those who sit nearby, but it has never been clear whether that was just social grooming. Now we know that real information is flowing. We keep being told that because of cheap, ubiquitous communication technology, distance is dead. But if there was ever a company that we should expect to exemplify that idea, surely it was Google. This research suggests that it is as important as ever to be sitting in the right place.

-

Reading Tim Harford, an objection popped up in my mind. If we had ben following this micro-geography model all along, then we would never have had open-source software (e.g., Linux, Mozilla FireFox, WordPress) and open knowledge (e.g., Wikipedia, group blog BoingBoing). In all these instances, collaboration has been done over the Internet by people who don’t know each other in person —although they happen to meet, once in a while . And yet these open projects have contributed greatly, in my view, to the making of our global digital civilization.

-

Enterprise prediction market consultant Jed Christiansen:

I’ve seen this in other markets I’ve run, as well. At the beginning traders are all over the place, with a fair bit of volatility and inaccuracy. But pretty quickly this settles out. I believe it’s a combination of “dumb” traders getting frustrated and stopping, and all traders becoming more sensitive to the risks they’re taking. [...]

Google used a CDA model for their markets. This created arbitrage opportunities, when the sum of the bid prices was more than $1, and when the sum of the ask prices was less than $1. The authors found 1,747 instances of the former, and 495 instances of the latter. As they noted, this demonstrates an aversion to short selling contracts. What was interesting to me was how the market reacted to this. The median duration of any of these arbitrage opportunities was just two minutes, demonstrating that they were correctable. Even better is this: “One trader in Google’s markets wrote a trading robot that was extremely prolific and ended up participating in about half of all trades. Many of these trades exploited arbitrage opportunities available from simultaneously selling all securities in a bundle.” All it takes is one person and a little programming knowledge to make the market much more efficient and liquid! I would never discourage this kind of behaviour. [...]

Bo Cowgill’s reacting about the organizational-behavior argument:

[...] This is not the first reaction along these lines. I am perplexed by the response. I can understand why other companies may not want to replicate our analysis of information flows. Perhaps it wouldn’t be worth the effort. Perhaps they would get identical results. And perhaps the company wouldn’t have the all the necessary data.

However, I expected that people could easily see value in the analysis of granular trade-by-trade data — especially if that data is joined with data about traders and outside events happening at the moment of the trades. We described one very generic application of this approach, but you can imagine much more actionable and company-specific ones.

I will mention one: The data contains real-time metrics on the distribution of knowledge and attitudes within a firm at a highly granular level. You can get metrics on for specific of the firm, for specific classes of employees and for specific topics. You can do this for either customers or employees, and have the metrics for any moment in time. The quality of these metrics will be extremely strong, because participants have been incentivized to reveal their true expectations.

Our analysis spoke in very general terms about the flow of information between Google employees — we don’t reference specific groups or draw distinctions between them — which is where a lot of actionable data was. Trade-by-trade data can reveal characteristics of specific working groups: What they know, how they feel, how they process and share information and how all of that changes over time. I didn’t try to put any of this in the paper because the conclusions would be sensitive, and I thought this application was pretty obvious to anybody who understood our methodology.

UPDATE: Our findings about the clustering of attitudes should also inform anyone who thinks that diversity is important for crowd-wisdom applications — as James Surowiecki famously suggests in The Wisdom of Crowds.

Our analysis suggests that groupthink primarily happens within language networks and small physical spaces (with social/professional networks playing a secondary role, and demographic networks playing a non-existent one). Remember that as you’re selecting your traders. If they already work/sit/chat together, the groupthink may already exist and the market won’t cure it.

Reacting on Jed Christiansen’s other post, Bo Cowgill adds:

I’ve also heard that other companies would find it impossible to analyze the interaction between their market and the organization. Why? Lack of data. Our analysis benefited from a wealth of internal data (including GPS coordinates of offices) that other companies don’t store.

You may be surprised at how much data average companies really have. For example, Google had social network surveys; many companies do not. However, many standard corporate applications (such as email, calendars, telephones and code reviews) contain implicit social networks that can be used in place of data gathered from surveys.

Or, consider this: I recently met with people from Google’s real estate management group. Turns out, they have records of the floorplans of Google’s offices in electronic format. Not only can someone use these records to find the distance between offices (without GPS coordinates) — you can also find the total area and perimeter of each office, which desks are open (cube-style) vs. enclosed, the walking distances between offices and more.

Surprised and impressed, I asked if it was typical for companies to have all of this information. The response was: “Any Fortune 1000 company would have this data about their offices.” Everyone in the room said his previous employer had the same data — typically managed through computer-aided facility management systems such as Archibus or Infor.

Bo Cowgill:

Some more remarks about applications that combine prediction markets and organizational data (org charts, social networks, seating locations). The obstacle to these applications is not a lack of data. Jed mentions privacy concerns — and if he thinks this is a big obstacle then I’d be interested in discussing his thoughts.

A bigger problem is that that current prediction market vendors and consultants cannot support these applications. At heart, these vendors are software engineers and salespeople at heart, not statisticians or data miners. They want to write one system that can support lots of clients. At conferences, one hears PM vendors complain about having to do “customization” work for clients.

This approach would not work for the applications I describe for two reasons:

1. The inputs for different clients won’t be the same. Each client’s organizational data will likely take a different structure. This makes it difficult for prediction market vendors to architect a single system that can served many clients (yet another challenge with integrating markets with other corporate IT services).
2. The outputs for different clients won’t be the same. The business relevance and statistical power of each analysis will differ with each client’s data.

Prediction market vendors may also need to familiarize themselves with the statistical learning methods necessary to fully utilize these rich datasets. So what’s the solution? First, move to a software-and-consulting model. By ‘consulting,’ I don’t mean ‘consulting on how to implement the market.’ I’m talking about helping the client solve its problem using a variety of data, including prediction market data.

Second, the vendors also need to pitch prediction markets as more than a forecasting tool. People in the business world commonly identify as data junkies — probably more so than they identify with the ‘wisdom of crowds’ ethos. It is unclear how much companies really care about accurate forecasting anyway.

-

Robin Hanson:

Info Value = the added accuracy the markets provide relative to other mechanisms, times the value that accuracy can give in improved decisions, minus the cost of maintaining the markets, relative to the cost of other mechanisms.

A highly accurate market has little value if other mechanisms can provide similar accuracy at a lower cost, or if few substantial decisions are influenced by accurate forecasts on its topic.

-

Bo Cowgill:

Although PM experiments make a company look ‘cool,’ disclosing specific high profile applications isn’t necessarily in the firm’s interests. I can vouch for this based on my own experience. Of a firm’s major stakeholders, we don’t know who would find a particular application unfair or irresponsible. This is particularly true if the application will create winners and losers — as would often be the case with high value markets.

Robin Hanson:

[...] I meant trying to field the highest value applications. That is naturally measured in accounting terms – value minus cost. Measures of popularity or familiarity would not at all be the same thing.

Bo Cowgill:

[...] I did not say anything about the popularity of the markets. I proposed measuring the number of employees working in a job that has a prediction market on it. This is a function of the firms decision to implement markets on a wide variety of relevant topics for the business.

I know this is a heuristic, and there would be greater value in trying to measure the value added in dollars. However, a monetization study is likely to be 1) more time consuming to produce, and 2) just as unreliable.

We aren’t likely to get honest answers from people by asking them to estimate the value of additional information at various specific hypothetical moments in the past. People are not good at making these type of estimations, especially when there is no incentive to get it right (and a lot of reasons to get it wrong).

On the costs side: Even if we had good data about how much individual employees were spending on the site browsing, etc — it is unclear how to price such their time. Is it work, or is it leisure? Should we model this as employees having an hourly rate (even though most are salaried)? I know you had some thoughts on this before which I don’t remember (do feel free to share), but I’m not convinced these issues can be resolved in a persuasive way.

Once people realize what’s going on with the methodology of such research, they’ll realize what a totally hackable and unreliable study it is — and it will lose its persuasive value at Google as well as externally.

In summary: Doing a value-of-information calculation on prediction markets itself does not seem to offer very much information value. Because of the low rigor, it would offer little additional persuasive value at a great cost.

Robin Hanson:

I can’t have much optimism about a business practice whose proponents aren’t even willing to try to offer a cost-benefit calculation. You could count how many employees had ever gone to a TQM meeting, but that wouldn’t tell you if TQM is valuable or not.

[Here's Robin Hanson's website. For your information (if you are a newbie), Robin Hanson is the most advanced researcher in the field of prediction markets. He co-invented the modern-day prediction markets, the concept of decision markets, and a new marked design, the Market Scoring Rule.]

-

Voila. Thanks for your attention. Feel free to publish a comment, just below… or blog about it —and I’ll update this post with some of your thoughts.

-

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

-

Usability expert Jakob Nielsen:

[...] Good practices include making a site easy to use, good search tools, the use of text free of jargon, usability testing and a consideration of design even before the first line of code is written. [...]

Research suggests that users of a site split into three groups. One that regularly contributes (about 1%); a second that occasionally contributes (about 9%); and a majority who almost never contribute (90%). [...]

Jakob Nielsen (the Pope of Internet usability) previously wrote:

All large-scale, multi-user communities and online social networks that rely on users to contribute content or build services share one property: most users don’t participate very much. Often, they simply lurk in the background. In contrast, a tiny minority of users usually accounts for a disproportionately large amount of the content and other system activity. This phenomenon of participation inequality was first studied in depth by Will Hill in the early ’90s, when he worked down the hall from me at Bell Communications Research (see references below). When you plot the amount of activity for each user, the result is a Zipf curve, which shows as a straight line in a log-log diagram. User participation often more or less follows a 90-9-1 rule:
- 90% of users are lurkers (i.e., read or observe, but don’t contribute).
- 9% of users contribute from time to time, but other priorities dominate their time.
- 1% of users participate a lot and account for most contributions: it can seem as if they don’t have lives because they often post just minutes after whatever event they’re commenting on occurs.