Using Prediction Markets to Track Information Flows: Evidence from Google – VIDEO – Bo Cowgill on Googles enterprise prediction markets – OReilly Money:Tech

No Gravatar

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

Via Daniel Horowitz (Business and Technology Consultant)

Blip.TV &#8212- (FLV file)

It&#8217-s cool. :-D

Google Web Search

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

ABSTRACT: In the last 2.5 years, Google has conducted the largest corporate experiment with prediction markets we are aware of. In this paper, we illustrate how markets can be used to study how an organization processes information. We document a number of biases in Google’s markets, most notably an optimistic bias. Newly hired employees are on the optimistic side of these markets, and optimistic biases are significantly more pronounced on days when Google stock is appreciating. We find strong correlations in trading for those who sit within a few feet of one another- social networks and work relationships also play a secondary explanatory role. The results are interesting in light of recent research on the role of optimism in entrepreneurial firms, as well as recent work on the importance of geographical and social proximity in explaining information flows in firms and markets.

DISCUSSION: In the past few years, many companies have experimented with prediction markets. In this paper, we analyze the largest such experiment we are aware of. We find that prices in Google’s markets closely approximated event probabilities, but did contain some biases, especially early in our sample. The most interesting of these was an optimism bias, which was more pronounced for subjects under the control of Google employees, such as would a project be completed on time or would a particular office be opened. Optimism was more present in the trading of newly hired employees, and was significantly more pronounced on and immediately following days with Google stock price appreciation. Our optimism results are interesting given the role that optimism is often thought to play in motivation and the success of entrepreneurial firms. They raise the possibility of a “stock price-optimism-performance-stock price” feedback that may be worthy of further investigation. We also examine how information and beliefs about prediction market topics move around an organization. We find a significant role for micro-geography. The trading of physically proximate employees is correlated, and only becomes correlated after the employees begin to sit near each other, suggesting a causal relationship. Work and social connections play a detectable but significantly smaller role.

An important caveat to our results is that they tell us about information flows about prediction market subjects, many of which are ancillary to employees’ main jobs. This may explain why physical proximity matters so much more than work relationships – if prediction market topics are lower-priority subjects on which to exchange information, then information exchange may require the opportunities for low-opportunity-cost communication created by physical proximity. Of course, introspection suggests that genuinely creative ideas often arise from such low-opportunity-cost communication. Google’s frequent office moves and emphasis on product innovation may provide an ideal testing ground in which to better understand the creative process.

PAPER BODY: In the last 4 years, many large firms have begun experimenting with internal prediction markets run among their employees. The primary goal of these markets is to generate predictions that efficiently aggregate many employees’ information and augment existing forecasting methods. […] In this paper, we argue that in addition to making predictions, internal prediction can provide insight into how organizations process information. Prediction markets provide employees with incentives for truthful revelation and can capture changes in opinion at a much higher frequency than surveys, allowing one to track how information moves around an organization and how it responds to external events. […]

We can draw two main conclusions. The first is that Google’s markets, while reasonably efficient, reveal some biases. During our study period, the internal markets overpriced securities tied to optimistic outcomes by 10 percentage points. The optimistic bias in Google’s markets was significantly greater on and following days when Google stock appreciated. Securities tied to extreme outcomes were underpriced by a smaller magnitude, and favorites were also overpriced slightly. These biases in prices were partly driven by the trading of newly hired employees- Google employees with longer tenure and more experience trading in the markets were better calibrated. Perhaps as a result, the pricing biases in Google’s markets declined over our sample period, suggesting that corporate prediction markets may perform better as collective experience increases.

The second conclusion is that opinions on specific topics are correlated among employees who are proximate in some sense. Physical proximity was the most important of the forms of proximity we studied. Physical proximity needed to be extremely close for it to matter. Using data on the precise latitude and longitude of employees’ offices, we found that prediction market positions were most correlated among employees sharing an office, that correlations declined with distance for employees on the same floor of a building, and that employees on different floors of the same building were no more correlated than employees in different cities.4 Google employees moved offices extremely frequently during our sample period (in the US, approximately once every 90 days), and we are able to use these office moves to show that our results are not simply the result of like-minded individuals being seated together. […]

Our findings contribute to three quite different literatures: on the role of optimism in entrepreneurial firms, on employee communication in organizations, and on social networks and information flows among investors. […]

The lessons of the literature informed Google CEO Eric Schmidt and Chief Economist Hal Varian’s (2005) third rule for managing knowledge workers: “Pack Them In.” Indeed, the fact that Google employees moved so frequently during our sample period suggests that considerable thought is put into optimizing physical locations. To this literature, which has largely relied on retrospective surveys to track communication, we illustrate how prediction markets can be used as high-frequency, market-incentivized surveys to track information flows in real-time. […]

Google’s prediction markets were launched in April 2005. The [Google prediction] markets are patterned on the Iowa Electronic Markets (Berg, et. al., 2001). In Google’s terminology, a market asks a question (e.g., “how many users will Gmail have?”) that has 2-5 possible mutually exclusive and completely exhaustive answers (e.g., “Fewer than X users”, “Between X and Y”, and “More than Y”). Each answer corresponds to a security that is worth a unit of currency (called a “Gooble”) if the answer turns out to be correct (and zero otherwise). Trade is conducted via a continuous double auction in each security. As on the IEM, short selling is not allowed– traders can instead exchange a Gooble for a complete set of securities and then sell the ones they choose. Likewise, they can exchange complete set of securities for currency. There is no automated market maker, but several employees did create robotic traders that sometimes played this role.

Each calendar quarter from 2005Q2 to 2007Q3 about 25-30 different markets were created. Participants received a fresh endowment of Goobles which they could invest in securities. The markets’ questions were designed so that they could all be resolved by the end of the quarter. At the end of the quarter, Goobles were converted into raffle tickets and prizes were raffled off. The prize budget was $10,000 per quarter, or about $25-100 per active trader (depending on the number active in a particular quarter). Participation was open to active employees and some contractors and vendors– out of 6,425 employees who had a prediction market account, 1,463 placed at least one trade. […]

Common types of markets included those forecasting demand (e.g., the number of users for a product) and internal performance (e.g., a product’s quality rating, whether a product would leave beta on time). […]

In addition, about 30 percent of Google’s markets were so-called “fun” markets –markets on subjects of interest to its employees but with no clear connection to its business (e.g., the quality of Star Wars Episode III, gas prices, the federal funds rate). Other firms experimenting with prediction markets that we are aware of have avoided these markets, perhaps out of fear of appearing unserious. Interestingly, we find that volume in “fun” and “serious” markets are positively correlated (at the daily, weekly, and monthly frequencies), suggesting that the former might help create, rather than crowd out, liquidity for the latter. […]

Google’s prediction markets are reasonably efficient, but did exhibit four specific biases: an overpricing of favorites, short aversion, optimism, and an underpricing of extreme outcomes. New employees and inexperienced traders appear to suffer more from these biases, and as market participants gained experience over the course of our sample period, the biases become less pronounced. […]

FOOT NOTE: One trader in Google’s markets wrote a trading robot that was extremely prolific and ended up participating in about half of all trades. Many of these trades exploited arbitrage opportunities available from simultaneously selling all securities in a bundle. In order to avoid having this trader dominate the (trade-weighted) results in Table 9, we include a dummy variable to control for him or her. None of the results discussed in the above paragraph are sensitive to removing this dummy variable.

APPENDIX:

Google Chart 1

Google Chart 2

Bo Cowgill&#8217-s precision point on micro-geography:

Below you can see a snapshot of trading in one of our offices. The areas where employees are making profitable decisions is green, and the areas where employees are making unprofitable decisions is red. There are about 16 profitable traders in that big green blotch in the middle!

Chart of the Day: Information Sharing at Google

More information from our previous blog post on the Google paper

Using Prediction Markets to Track Information Flows: Evidence from Google – (PDF file – PDF file) – by Bo Cowgill, Justin Wolfers, and Eric Zitwewitz – 2008-01-06

John McCains grumpy old pal writes to Bo Cowgills boss.

No Gravatar

Joe Lieberman:

May 19, 2008

Dr. Eric Schmidt
Chairman of the Board and Chief Executive Officer
Google, Inc.

1600 Amphitheatre Parkway
Mountain View, CA 94043

Dear Dr. Schmidt:

YouTube is being used to share videos produced by al-Qaeda and other Islamist terrorist groups. The purpose of this letter is to request that Google implement its own policy against this offensive material, remove these videos from YouTube, and prevent them from reappearing.

Today, Islamist terrorist organizations rely extensively on the Internet to attract supporters and advance their cause. The framework for much of this Internet campaign is described in a bipartisan staff report released last week by the Senate Committee on Homeland Security and Governmental Affairs (“Committee”), which I am privileged to chair, titled Violent Islamist Extremism, the Internet, and the Homegrown Terrorist Threat. The report explains, in part, how al-Qaeda created and manages a multi-tiered online media operation that produces content intended to enlist followers in countries all over the world, including the United States. Central to this media campaign is the branding of content with an icon or logo to guarantee authenticity that the content was produced by al-Qaeda or allied organizations like al-Qaeda in Iraq, Ansar al-Islam (a.k.a Ansar al-Sunnah) or al-Qaeda in the Land of the Islamic Maghreb. All of these groups have been designated Foreign Terrorist Organizations (FTO) by the Department of State.

Searches on YouTube return dozens of videos branded with an icon or logo identifying the videos as the work of one of these Islamist terrorist organizations. A great majority of these videos document horrific attacks on American soldiers in Iraq or Afghanistan. Others provide weapons training, speeches by al-Qaeda leadership, and general material intended to radicalize potential recruits.

In other words, Islamist terrorist organizations use YouTube to disseminate their propaganda, enlist followers, and provide weapons training – activities that are all essential to terrorist activity. According to testimony received by our Committee, the online content produced by al-Qaeda and other Islamist terrorist organizations can play a significant role in the process of radicalization, the end point of which is the planning and execution of a terrorist attack. YouTube also, unwittingly, permits Islamist terrorist groups to maintain an active, pervasive, and amplified voice, despite military setbacks or successful operations by the law enforcement and intelligence communities.

YouTube posts “community guidelines” for users to follow, but it does not appear that the company is enforcing these guidelines to the extent they would apply to this content. For example, the community guidelines state that “[g]raphic or gratuitous violence is not allowed. If your video shows someone getting hurt, attacked, or humiliated, don’t post it.” Many of the videos produced by one of the production arms of al-Qaeda show attacks on U.S. forces in which American soldiers are injured and, in some cases, killed. Nevertheless, those videos remain available for viewing on YouTube. At the same time, the guidelines do not prohibit the posting of content that can be readily identified as produced by al-Qaeda or another FTO.

I ask you, therefore, to immediately remove content produced by Islamist terrorist organizations from YouTube. This should be a straightforward task since so many of the Islamist terrorist organizations brand their material with logos or icons identifying their provenance. In addition, please explain what changes Google plans to make to the YouTube community guidelines to address violent extremist material and how Google plans to enforce those guidelines to prevent the content from reappearing.

Protecting our citizens from terrorist attacks is a top priority for our government. The private sector can help us do that. By taking action to curtail the use of YouTube to disseminate the goals and methods of those who wish to kill innocent civilians, Google will make a singularly important contribution to this important national effort.

Thank you for your immediate attention to this critical matter and I look forward to your response.

Sincerely,

Joseph I. Lieberman (ID-CT)
Chairman, Senate Committee on Homeland Security and Governmental Affairs

Prediction markets = the future of journalism -said, from day one, Emile Servan-Schreiber of NewsFutures. Emile, if you have balls, lets do it -all together.

My yesterday&#8217-s post about the Obama&#8211-Clinton prediction markets was the most popular Midas Oracle story of that Monday. Hummmm&#8230- No idea why&#8230- I was not helped by Google Search or by an external blogger. Sounds like our Midas Oracle web readers and feed subscribers liked it &#8230- for some reasons I have yet to discover fully.

Anyway.

  1. I&#8217-m minding a grand &#8220-Midas Oracle Project&#8220-, and you can join it.
  2. Emile believes that prediction markets represent &#8220-the future of journalism&#8220-. I am trying to mind, specifically, what form could take the &#8220-prediction market journalism&#8220-.
  3. The idea is this: We need to put the charts of prediction markets inside news stories, and those stories should incorporate the meaning of the probability fluctuations (a la Justin Wolfers).
  4. If we stay in our armchairs, nothing will happen, because most of the old-school journalists and bloggers don&#8217-t think much of the prediction markets. The prediction market infiltration in the Mediasphere and the Blogosphere is like a weak stream, right now. I don&#8217-t have the patience to wait until &#8220-2020&#8243-.
  5. I don&#8217-t think that much will come out of the prediction exchanges. The BetFair blog and the InTrade newsletter are 2 pieces of crap &#8212-they compete in content quality with the Mongolian edition of the News Of The World.
  6. If you look at the evolution of the media, you see that the old-school, dead-tree publications are slowly dying, and are replaced by professional blog networks &#8212-look especially in the IT industry, with TechCrunch, etc. What you have is writers who publish only for the Web, and who fill a vertical niche. (And, the Washington Post is now publishing content from&#8230- guess who.)
  7. Needless to say, prediction market journalism is costly. Now, go directly to point #8, because that&#8217-s where the beef is.
  8. Yes, I have &#8220-heard of Christmas&#8221- :-D , and I understand Robin Hanson&#8217-s reasoning. [*] That&#8217-s where my funding idea lays. The idea is to think hard about who &#8220-might actually be willing to pay&#8221-. I am thinking of a class or organizations that &#8220-might actually be willing to pay&#8221-, provided 2 things. Number one, that I operate a certain twist on my form of prediction market journalism. Number two, that this project becomes the project of many prediction market people, or, better, of the whole prediction market industry &#8212-not just Chris Masse&#8217-s one. Those 2 things are essential.
  9. So, Emile, wanna join the &#8220-Midas Oracle Project&#8220-?

[*] APPENDIX:

The &#8220-high IQ&#8221- Robin Hanson:

Chris, you’ve heard of Christmas I presume. Many people circulate lists of items they might like for Christmas. If you did, would you circulate a list of million franc/dollar gift ideas for people to give you? Would you consider that list more honest/logical than a list of gifts of roughly the price you think others might actually be willing to pay?

Prediction Markets at Google – by Peter A. Coles, Karim R. Lakhani, Andrew McAfee

No GravatarAlas, that paper is not free to access.

Andrew McAfee&#8217-s post reveals this:

Prediction markets were (sic) very much like stock markets. They contained securities, each of which had a price. [&#8230-]

Not sure why they used the past tense.

Prediction markets are in fact event derivative markets.

Papers from Robin Hanson, Justin Wolfers, Eric Zitzewitz, Koleman Strumpf, etc., are free to download.

Via George Tziralis, of Ask Markets.

Previously: Do Google’s enterprise prediction markets work?

Previous blog posts by Chris F. Masse:

  • “Annette 15”, the once-hot female poker star sponsored by BetFair Poker, does blog only twice a month on the official BetFair blog… when she blogs at all… if you call that blogging.
  • Inkling Markets bring in awards, honors, advisors, and new clients —leaving competition in the dust.
  • No need of enterprise prediction markets to boost intra-corporation communication
  • Inkling Markets is included in the 2008 list of “Cool Vendors” by Gartner.
  • BetFair-TradeFair has won its second Queen’s Award for Enterprise in its eight-year history.
  • Inkling Markets is one of the “Hot Companies To Watch In 2008”, according to Forrester.
  • Plenty of great news coming from Inkling Markets in the coming weeks

The Promise Of Enterprise Prediction Markets – The McKinsey conference should have been rooted in the economic science and McKinsey should have invited economists.

No GravatarMcKinsey: The Promise Of Prediction Markets

James Surowiecki: The premise is that under the right circumstances, the collective judgment of a large group of people will generally provide a better picture of what the future might look like than anything one expert or even a small group of experts will come up with. [&#8230-]

James Surowiecki: The Wisdom of Crowds is not an argument against experts. It is saying that you shouldn’t rely wholly on the judgment of one person or even a very small group of people. But for a crowd to be smart, it needs to satisfy certain criteria. It needs to be diverse, so that people are bringing different pieces of information to the table. It needs to be decentralized, so that no one at the top is dictating the crowd’s answer. It needs to summarize people’s opinions into one collective verdict. And the people in the crowd need to be independent, so that they pay attention mostly to their own information and don’t worry about what everyone around them thinks.

James Surowiecki: [&#8230-] One shortcoming is that a lot of people inside organizations don’t find the market mechanism intuitive or easily understood. They find it very challenging to use, which limits the pool of people who participate.

On James Surowiecki&#8217-s last remark, I would say that Robin Hanson&#8217-s MSR technology (which powers most enterprise prediction exchanges but Google&#8217-s one) brought much needed simplification to trading.

Overall, a good roundup, but the conference speakers should have mentioned Robin Hanson&#8217-s pioneering work, and McKinsey should have invited him. He would have towered anybody and given great insights.

See Jed Christiansen for other remarks.

As an aside, I&#8217-d say I prefer the sketch that is supposed to represent Bo rather than the real photo. The sketch makes him look like he is subtitle, charming, smiling, humble, and modest &#8212-quite a quantum leap. :-D

Bo Cowgill

Bo Cowgill – Economics at Google

  • PhotoShop designers improve the look of models on glossy magazine covers.
  • Sketchy artists improve the look of testosteroned, ultra-serious, ambitious, young business managers. :-D

Previously: Do Google’s enterprise prediction markets work?

Previous blog posts by Chris F. Masse:

  • Collective Error = Average Individual Error – Prediction Diversity
  • When gambling meets Wall Street — Proposal for a brand-new kind of finance-based lottery
  • The definitive proof that it’s presently impossible to practice prediction market journalism with BetFair.
  • The Absence of Teams In Production of Blog Journalism
  • Publish a comment on the BetFair forum, get arrested.
  • If I had to guess, I would say about 50 percent of the “name pros” you see on television on a regular basis have a negative net worth. Frightening, I know.
  • You can’t measure the usefulness of a system by how many resources it consumes.

The New York Times article doesnt mention Googles enterprise prediction markets, alas. – Bo Cowgill says that the illustration published in the sidebar defines exclusively what is done at Google.

No Gravatar

Right-click on the New York Times graphic below, open Bo Cowgill&#8217-s post in another browser tab, and read his arguments.

NYT PMs

Image Credit: Chris Gash for the New York Times

Adam Siegel of Inkling Markets is also out with a post on that NYT article, but it is of no intellectual interest. Maybe Adam should blog less quickly and eat more fish.

I forgot to tell you, the other day, that Best Buy is a Consensus Point client, but you knew that already.

Previously: The New York Times is telling the business world that enterprise prediction markets are an essential management tool.

[Via Xpree]

Previously: Do Google’s enterprise prediction markets work?

Previous blog posts by Chris F. Masse:

  • The Terror Finance Blog
  • Playing fantasy sports is not gambling. The Unlawful Internet Gambling Enforcement Act includes a specific exemption for fantasy sports, provided the prizes are determined in advance and the imaginary teams don’t correspond to any real teams.
  • Inkling Markets’ Advisory Board… which does not want to tell its name
  • BetFair created the world’s largest ad banner —as certified by the Guinness Book of World Records.
  • Why Emile Servan-Schreiber is on to something with Bet 2 Give —and why InTrade, TradeSports and BetFair should each have a philanthropy wallet.
  • The CFTC is going to close the comments in 14 days. We have 14 days left to convince the CFTC to accept FOR-PROFIT prediction exchanges, and counter the evil petition organized by the American Enterprise Institute (which has on its payroll Paul Wolfowitz, the bright masterminder of the Iraq war).
  • The purpose of X2 is to identify future disruptions, opportunities, and competitive landscapes related to the content and dynamics of global science and technology innovation- to develop a new platform for understanding global innovation trends- and to present this information to policy- and decision-makers, as well as the general public, in a useful form.

Launch a prediction market startup for free thanks to Google App Engine.

No Gravatar

Run your web applications on Google&#8217-s infrastructure.

Matt Cutts:

Google launched App Engine, which lets you write code for a web application, then Google takes care of the scaling/failover/logistics-type issues. You can store your data in a Google Bigtable using the Google File System (GFS). There’s a bunch of App Engine APIs to simplify things like sending email and fetching urls. Your application can authenticate users that are using Google Accounts, so you can avoid the whole “ask your users to create a new account” issue if you want.

O&#8217-Reilly Radar

Google App Engine

[Competitor: Amazon AWS]

YouTube Video

Bo Cowgill, this is completely crazy.

UPDATE: Raves from TechCrunch and Silicon Alley Insider.

UPDATE: Google AppEngine – A Second Look

UPDATE: Praxy