July 9th, 2008

-
1:25pm Opening Remarks

-

1:30 – 2:20pm: Session 1: Practical Applications and Experiences

-

Speaker: Emile Servan-Schreiber (1:30-1:45pm)

• Keys to successful implementation of an internal corporate prediction market (Emile Servan-Schreiber)

-

Speaker: Todd Henderson (1:45-2:05pm)

• Prediction Markets for Corporate Governance (Michael Abramowicz and Todd Henderson)
• Predicting Crime (Todd Henderson, Justin Wolfers, and Eric Zitzewitz)

Discussion (2:05pm – 2:15pm)
-

2:20pm-3:10pm Session 2: Security Design and Theory

-

Speaker: Hajime Mizuyama (2:15 pm — 2:30pm)

• Intra-Firm Prediction Market System Using VIPS for Demand Forecasting (Hajime Mizuyama)

-

Speaker: Stefan Stathel (2:30 pm –2:45pm)

• Information Efficiency and Liquidity in Information Markets: A market-maker based approach (Stefan Stathel, Clemens van Dinther, and Stefan Luckner)

-

Speaker: Eiichero Kazumori (2:45 pm — 3:00pm)

• A Strategic Theory of Markets (Eiichiro Kazumori)

Discussion (3pm-3:10pm)
-

3:10pm-3:30pm: Break

-

3:30-4:25pm: Session 3: Event Detection

-

Speaker: Todd Kuethe (3:30 – 3:45pm)

• Identifying “Tipping Points” in US Presidential Campaigns: Evidence from Prediction Market Prices 1992-2004 (Todd H. Kuethe, David Ubilava, Benoit Delbecq, and Kenneth A. Foster)

-

Speaker: George Tziralis (3:45pm – 4:00pm)

• Detecting Important Events using Prediction Markets, Text Mining, and Volatility Modeling (George Tziralis and Panos Ipeirotis)

-

Speaker: Abe Othman (4:00pm-4:15pm)

• Time Inconsistency and Uncertainty Aversion in Information Markets (Ivan Corwin and Abe Othman)

Discussion (4:15pm-4:25pm)

-
Session 4: Comparison to Alternatives (4:25-4:55pm)

-
Speaker: Andreas Graefe (4:25-4:45pm)

• Can you beat the market? Accuracy of Individual and Group Post-Prediction Market Judgments (Andreas Graefe and J. Scott Armstrong)
• Group Decision Making – Face-to-face Meetings, Nominal Groups, Delphi, and Prediction Markets Compared (Andreas Graefe and J. Scott Armstrong)

Discussion: 4:45-4:55pm

-

Wrap-up: 4:55-5pm

IIF’s SIG on Prediction Markets

Research Papers

Basics

Several studies explain the concept of prediction markets and provide useful summaries of the method, e.g.

- Spann, M. & Skiera, B. (2003). Internet-based Virtual Stock Markets for Business Forecasting, Management Science, 49, 1310-1326. [Full text]
- Wolfers, J. & Zitzewitz, E. (2006). Prediction Markets in Theory and Practice, New Palgrave Dictionary of Economics and the Law (in press). [Full text]
- Wolfers, J. & Zitzewitz, E. (2004). Prediction Markets, Journal of Economic Perspectives, 18, 107-126. [Full text]
- An overview and classification of 152 studies on prediction markets, published between 1991 and 2006, is provided by
Tziralis, G. & Tatsiopoulos (2007). Prediction Markets: An Extended Literature Review, Journal of Prediction Markets, 1, 75-91. [Full text]

Evidence on the accuracy of prediction markets

This section summarizes research that analyzes the relative performance of prediction markets and other forecasting methods.

Markets vs. polls (election forecasting)

- Berg, J., Nelson, F. & Rietz, T. (2008). Prediction Market Accuracy in the Long Run, International Journal of Forecasting, 24, 283-298. [full text]
- Erikson R. S. & Wlezien C. (2007). Are Political Markets Really Superior to Polls as Election Predictors? Public Opinion Quarterly, forthcoming. [full text]
- Stix, G. (2008): When Markets Beat the Polls, Scientific American Magazine, March 2008. [Abstract]

Markets vs. unaided experts and groups

- Pennock, D. M., Lawrence, S., Giles, C.L. & Nielsen, F.A. (2000). The Power of Play: Efficiency and Forecast Accuracy in Web Market Games, Technical Report 2000-168, NEC Research Institute. [full text]
- For predicting Oscar Award winners, Pennock et al. (2000) compared prices of the Hollywood Stock exchange to expert judgments of five movie columnists. On the day the experts revealed their forecasts, only one of them was better than the market predictions. From the day after, the market outperformed all experts as well as the expert consensus.
- Servan-Schreiber, E. J., Wolfers, J., Pennock, D. M. & Galebach, B. (2004). Prediction Markets: Does Money Matter? Electronic Markets, 14, 243-251. [full text]
- For predicting the results of NFL games, Servan-Schreiber et al. (2004) compared the forecasts of two markets to those of 1,947 self-selected individuals. At the end of the season, the markets ranked 6th and 8th compared to the individuals. The human average – which would be the outcome of a classical survey – ranked 39th.

Markets vs. other forecasting methods

- Chen, K. Y., Plott, C. R. (2002). Information Aggregation Mechanisms: Concept, Design and Implementation for a Sales Forecasting Problem, Social Science Working Paper No.1131, California Institute of Technology, Pasadena. [full text]
- For forecasting sales figures, Chen and Plott (2002) reported on an internal market at Hewlett-Packard that beat the official forecasts of the company in 6 out of 8 events.
- Jones Jr., R. J. (2008). The state of presidential election forecasting – The 2004 experience, International Journal of Forecasting, 24, 308-319. [Abstract]
- Jones (2008) analyzed the forecasts of IEM’s vote-share market for the 2004 election and compared them to traditional polls, a Delphi expert survey, regression models and a combination of all four approaches, the Pollyvote. He concludes that in comparison with most methods of forecasting the popular vote, the IEM was the superior performer.Spann, M. & Skiera, B. (2003). Internet-based Virtual Stock Markets for Business Forecasting, Management Science, 49, 1310-1326. [Full text]
- Spann and Skiera (2003) compared forecast accuracy of an internal market at a large German mobile phone operator. They found that the market forecasts outperformed were more accurate than four extrapolation models (arithmetic mean, geometric mean, linear trend and exponential trend).

Corporate Markets

- Chen, K.-Y. & Plott, C. R. (2002). Information Aggregation Mechanisms: Concept, Design and Implementation for a Sales Forecasting Problem. Social Science Working Paper No.1131, California Institute of Technology, Pasadena. [Full text]
- Cowgill, B., Wolfers, J. & Zitzewitz, E. (2008). Using prediction markets to Track Information Flows: Evidence from Google, working paper. [Full text]
- Ortner, G. (1997). Forecasting Markets – An Industrial Application: Part I, working paper, TU Vienna. [Full text]
- Spann, M. & Skiera, B. (2003). Internet-based Virtual Stock Markets for Business Forecasting, Management Science, 49, 1310-1326. [Full text]

Decision Markets

- Hanson, R. (1999). Decision Markets, IEEE Intelligent Systems, 14, 16-19.

Manipulation

- [Except] Hansen et al. (1998), most empirical studies report that manipulative attacks on result accuracy have not been successful historically (Rhode and Strumpf 2006), in the laboratory (Hanson et al. 2006), and in the field (Camerer 1998).
- Camerer, C. (1998): Can Asset Markets Be Manipulated? A Field Experiment with Racetrack Betting, Journal of Political Economy, 106(3), 457-482. [Abstract]
- Hansen, J., Schmidt, C. & Strobel, M. (2004). Manipulation in Political Stock Markets – Preconditions and Evidence, Applied Economics Letters, 11, 459-463. [Abstract]
- Hanson, R., Oprea, R. & Porter, D. (2006). Information Aggregation and Manipulation in an Experimental Market, Journal of Economic Behavior & Organization, 60, 449-459. [full text]
- Rhode, P. W., and Strumpf, K. S. (2006). Manipulating Political Stock Markets: A Field Experiment and a Century of Observational Data, Working Paper, University of North Carolina(2006). [full text]

-

More research papers on prediction markets

-

The world’s largest steel maker.

CORNING
Display Technologies.

Japan’s largest advertising firm.

Voted “most innovative” pharmaceutical company in Fortune‘s 2003 and 2004 rankings.

The US packaged foods giant.

Giant U.S. Pharmaceutical.

Leading provider of objective analysis and effective solutions.

One of the world’s leading providers of outsourcing and IT services.

Europe’s leader in corrugated packaging.

Germany’s global powerhouse in electrical engineering and electronics.

Texas Department of Transportation.

The most complete source for integrated information and technology applications in the global financial services industry.

The No. 1 Internet brand globally.

External Links: RAND – Wikipedia on RAND

The achievements of RAND stem from its development of systems analysis. Important contributions are claimed in space systems and the United States’ space program, in computing and in artificial intelligence. RAND researchers developed many of the principles that were used to build the Internet. Numerous analytical techniques were invented at RAND, including dynamic programming, game theory, the Delphi method, linear programming, systems analysis, and exploratory modeling. RAND also pioneered the development and use of wargaming. Current areas of expertise include: child policy, civil and criminal justice, education, environment and energy, health, international policy, labor markets, national security, infrastructure, energy, environment, corporate governance, economic development, intelligence policy, long-range planning, crisis management and disaster preparation, population and regional studies, science and technology, social welfare, terrorism, arts policy, and transportation. RAND designed and conducted one of the largest and most important studies of health insurance. The RAND Health Insurance Experiment, funded by the then-U.S. Department of Health, Education and Welfare, established an insurance corporation to compare demand for health services with their cost to the patient. According to the 2005 annual report, “about one-half of RAND’s research involves national security issues.”

(((A competitor of NewsFutures has for client a very big non-profit organization.)))

While in the Delphi a small number of selected experts are asked to judge predefined theses, it is the goal of TechForX to enhance traditional foresight activities by involving a preferably broad public. Thus, TechForX invites every internet user to participate by revealing his / her opinion on a subset of the same theses that are asked in the Delphi.

The results will be incorporated in EPIS to improve strategic decision-making of European policy makers. Furthermore, the findings will be analyzed in my dissertation which aims at comparing the applicability of prediction markets for long-term developments to established approaches like the Delphi.

Your support for this research project by registering and participating at www.techforx.org is very much appreciated!

To get an impression on the discussed topics, here are some of the theses that are currently traded on TechForX:

• More money is spent on internet advertising in search engines and navigation tools than on television advertising.
• On average, users spend more time watching personalized TV on demand than fixed programs.
• A value added tax (VAT) for transactions within virtual worlds (like Second Life) is raised.
• Online self-publication of books (without involving publishing houses) is the predominant way of commercial distribution, even for established authors.
• Virtual visits to museums over the internet are more popular than physical visits.

Longtime readers are familiar with my views on prediction markets. I believe they have some value, when applied correctly. Where prediction markets excel is in acting as a collective real time polling mechanism for their participants. The closer the collective group is to the broader population, the more accurate these markets tend to be. Where they do poorly is when the collective attempt to “pool their ignorance” and forecast the random or the unknowable.

There is strength in any mechanism that can factor a large pool of participants collective experience and knowledge, discount several possibilities into the most likely outcome. These market mechanisms are hardly the Oracles of Delphi their supporters make them out to be. It is only by din of Humans’ even worse forecasting abilities (with experts leading the way down) that these markets garner so much respectability in the first place. Blame fuzzy thinking for placing too much credibility on a mechanism with so mixed a track record. There is also a subtle but important distinction between forecasting the future versus discounting various outcomes.

Which leads me to [the] New York Times column. In a bizarre twist of logic, the massive failure of the prediction markets in the US 2006 mid-term elections somehow gets credit for being right. They were not a day late and a dollar short, they were completely, totally and incontrovertibly wrong.

Except in the NY Times:

Consider the following: As of 11:50 p.m. on the evening of Election day, with the voting completed (except for Hawaii) and the majority of the ballots counted, Intrade gave the Republicans an 85 percent chance of retaining the Senate. Of course, we know that’s not at all what happened. If that’s not a forecasting failure, then what is?

If forecasting the results of an election held yesterday is considered prescience, then sign me up — and find me a bookie who will take my Superbowl bets on the Monday after the game. I suspect I will at least cover the spread.

To reiterate my views, these markets — thinly traded, easily manipulated, poorly administered — do have some real value. However, they have hardly been “remarkably clairvoyant,” as the Times describes them. Consider these recent acts of “clairvoyance:”

- GOP Retention of Senate
- Michael Jackson Trial Results
- Morgan Stanley CEO Purcell resignation
- Howard Dean’s Iowa Primary
- Pre-Election 2004
- Election day trading frenzy

There are some interesting explanations for some of these errors: Pooling the collective ignorance of millions does not produce Wisdom. The Michael Jackson Trial and Morgan Stanley’s CEO are perfect examples of that.

As to the prediction market’s failure of the Howard Dean primary and the 2006 mid-term elections, I’ve been toying with a theory. Since the political affiliation of Wall Street tends toward Republican, and since there is a big overlap between Wall Street and the participants of the prediction markets, the collective had an inherent bias built into it. Remember, where these markets excel is when they act as a realtime polling mechanism of their particpants. If the pool has a bias, the outcome may very well too. Hence, these widescale failures.

—

Bottom line: While having value, prediction markets are subject to error and bad outcomes. Some of this is due to their relative thinness of the markets; some is due to the inherent biases of the participants, and their failure to parallel the population at large.

They should be considered with the recognition of their fallibility.

—

Source:
Odds Are, They’ll Know ’08 Winner
DAVID LEONHARDT
New York Times, February 14, 2007
Economix

http://www.nytimes.com/2007/02/14/business/14leonhardt.html

—

Cross-Posted from The Big Picture

These market mechanisms are hardly the Oracles of Delphi their supporters make them out to be.

Barry Ritholtz is entitled to his opinion —cherry-picking the prediction market failures and forgetting the successes. I prefer listening to Justin Wolfers, Eric Zitzewitz, Robin Hanson, Paul Tetlock, Koleman Strumpf, David Pennock, etc. They are great economists who have been studying the accuracy of prediction markets for years, and their opinion of them is largely positive.

Addendum: Event though I disagree with Barry Ritholtz on that one, I want to point out to my readers that I view him as a very correct man and a great blogger.

Addendum #2: Excerpt of the Justin Wolfers comment…

[...] Yes, markets are fallible. But I still prefer the least fallible forecasting institution. And in many domains, that appears to be prediction markets.

Addendum #3: If you want more of Barry Ritholtz…

- The Infallibility of Markets

- The Message of the Markets

#1. I fully agree that vertical prediction exchanges (a la HSX) are performing a better service than vertical lines in a generalist prediction exchange. (I had some thoughts on that, last year.)

#2. The vertical prediction exchange that Alex Kirtland is talking about has popped up on Delicious and other places, because, yesterday, its founder has asked the beta users to help him promote the exchange.

#3. The founder of that “new” prediction exchange is already on the Midas Oracle list of blog authors, and will soon blog about it —the technical specifics are of interest.

#4. As for the other new “prediction market” cited by Alex Kirtland (Farmetrics), as I said the other day, I would be prudent about calling it a prediction exchange. Just because somebody writes a P.R. output with the keyword “prediction market” and places it at Google News does not mean that it is a prediction exchange, as we know it. The same remark goes for Numeria. Its founder uses the keyword “prediction markets” and paid the other guy $2,500 to be lined up in a prediction market conference, but that’s not the guarantee that what he promotes is of interest to the prediction market community. Actually, Numeria seems to have to do more with the Delphi method, and the linkage he wants to make with prediction markets is just that… an idea —just like thousands of other people have suggestions about prediction markets (sometimes interesting, sometimes unfeasible –and I include myself in the crowd). As for me, I’m being more and more prudent about what and who I index as “prediction markets”. (((I could be damn wrong, sometimes, of course. I’m just one observer, and there are plenty of others —smarter than I am.)))

The costs of building coalitions are also a neglected element in the theory of organizations. Even in the private sector, once we consider cohesion and morale, businesses have many fewer degrees of freedom than we might think. That is why merit pay and prediction markets are not as common as an economist might expect. Too often those institutions put people at odds with each other.

My Answer:

#1. Economists are poor innovators and lame marketers. Their “expectations” are necessarily always out of the ball park. (Note: I view Robin Hanson as an inventor, but maybe not as an innovator. And, anyway, Robin Hanson is “unusual”. He is an institution designer; the others economists are data analysts.)

#2. The field of prediction markets lacks a bunch of good Guy Kawasaki-type professionals (i.e., clever marketers).

#3. You shouldn’t expect the Planet Earth’s whole population to spend nights and days frantically and frenetically trading contracts on prediction markets. Only a small portion of the world’s people will play this game, and only a few topics will be of interest to this small group of prediction market traders. In the global forecasting industry (polls, predictive models, Delphi method, etc.), the market-generated prediction providers will always be tiny (if not microscopic). Think of the Hawaii islands in the Pacific Ocean.