Midas Oracle .ORG » Mechanism Designs

Dear MO reader: Why you should try Predictalot

David Pennock — Tue, 16 Mar 2010 15:21:49 +0000

Why should you try Predictalot?

Gamers: Make almost any prediction you can think of about March Madness, the NCAA men’s basketball tournament.
Sports fans: Check the crowd’s odds: Is St. Mary’s the next Cinderella?
Economists: Play with a true combinatorial prediction market with 9.2 quintillion outcomes and a single pool of liquidity, unlike almost any other of today’s financial and prediction markets.
Geeks: Ponder some of the interesting computer science challenges, including approximating #P-hard problems and an eerily similar sampling problem as faced by physicists.
Everybody’s doing it.
Barack Obama might do it, according to VentureBeat on NYTimes.com: “President Barack Obama will likely be busy this week [but]… maybe he’ll be able to sneak a peek at Predictalot on his BlackBerry between meetings.”

Midas Oracle = About + Archives + Authors + Best + Contact + Information Technology + Links + Probabilistic Predictions + Post-Mortem Predictions + Prediction Exchanges + Prediction Software + Publish Your Post + How To Publish

Predictalot = combinatorial prediction markets (a la Robin Hanson)

Chris F. Masse — Fri, 05 Mar 2010 19:36:37 +0000

Predictalot

David Pennock explains Predictalot.

The Prediction Market Police

Chris F. Masse — Fri, 28 Aug 2009 05:32:16 +0000

I should have listened to Panos Ipeirotis…!!!…

CrowdCast = market mechanism = binary spreads with a market maker

Chris F. Masse — Mon, 20 Jul 2009 19:31:30 +0000

Leslie Fine (CrowdCast Chief Scientist) to me:

Actually, our mechanism is a market, it’s just not a stock market. We use an automated market maker to efficiently price every bet, adjust crowd beliefs, and price an interim sell. In essence, participants trade binary spreads with the market maker.

Because our new version was not yet market-ready, I did not enter the markets vs. non-markets debate when you were having it some months ago. However, among other reasons, we avoid collective forecasting because it is too similar to collaborative forecasting, which is key in supply chain. Honestly, when all is said and done, our clients care not what the mechanism is. They care that we can efficiently gather team intelligence and translate it into actionable business intelligence. That is our mission.

CrowdCast website

Previously: CrowdCast = Collective Forecasting = Collective Intelligence That Predicts

CrowdCast = Collective Forecasting = Collective Intelligence That Predicts

Chris F. Masse — Thu, 25 Jun 2009 16:48:55 +0000

The fine people at CrowdCast (Mat Fogarty and Leslie Fine) are finally out today with their brand-new, no-trading, collective forecasting mechanism. The purpose is to aggregate information across one organization so as to generate the most objective business forecasts. The readers of Midas Oracle are well aware of the (relative) performance of the prediction markets (which are the most complex of all collective forecasting mechanisms). Here’s CrowdCast’s view on prediction markets:

In the Iowa Electronic Market (IEM) people bet on the outcomes of political races. On average, IEM has 23% less error than traditional polling methods.

The Hollywood Stock Exchange (HSX) public market lets people forecast Oscar winners and opening box office receipts. Its 700,000 active traders have predicted 92% of the Oscar winners in the last 3 years.

But, as you all know, Leslie Fine (their Chief Scientist) has decided to ditch the prediction market mechanism in favor of a simpler one.

We all knew that we had to start from scratch, and rebuild a mechanism that was easy to use, expressive both in terms of the question one can ask and the message space in which one can answer, and provided a high level of user engagement. We have abandoned the MSR [Market Scoring Rules] in favor of a new method that users are already finding much simpler and that requires a lower level of participation and sophistication than the usual stock market analogy.

So, CrowdCast is unveiling today their non-market solution:

With Crowdcast you get more accurate forecasts faster:

* Knowledge is dispersed throughout an organization, so you can invite many participants. Or you can be more selective. The Crowdcast market mechanism was designed to support both.

* Knowledge is weighted—participants can vary the level of their bet, depending on their confidence. It is also expressed as a confidence interval, rather than a point, which mirrors how people think and gives you actionable data.

* Accuracy is rewarded. As forecasts are realized, bets are closed out, winners are recognized

* Results are astoundingly accurate

Here’s a video that explains it all (translating “employee intelligence into enterprise intelligence”):

CrowdCast outputs both SaaS and consulting:

CrowdCast is in the New York Times today:

During a pilot period, five large companies, including Warner Brothers and General Motors, have been using Crowdcast to predict revenue, ship dates or new products from competitors. About 4,000 bets that have been placed, and predictions have been about 50 percent more accurate than official forecasts, Ms. Fine said.

At a media company with a new product to ship, 1,200 employees predicted a ship date and sales figures that resulted in 61 percent less error than executives’ previous prediction, according to Crowdcast. A pharmaceutical company asked a panel of scientists and doctors to predict regulatory decisions and new drug sales using Crowdcast and they were more accurate than the company’s original prediction 86 percent of the time.

CrowdCast’s PR release (boring stuff that Leslie sent me this early morning):

CROWDCAST LAUNCHES PREDICTION MARKETS PLATFORM FOR ENTERPRISE FORECASTING

Crowdcast Shakes Up the Forecasting Game

Redwood City, CA, June 25, 2009

Crowdcast, the leader in innovative corporate forecasting software, today announced the public availability of its prediction markets platform. By giving employees a marketplace where they place bets on important corporate metrics, Crowdcast reveals information that would normally not rise to executive awareness. The resulting forecasts are cleansed of bias and politics, and are significantly more accurate than traditional forecasts.

Forecasting is a multi-million dollar problem. According to the CFO Magazine/Duke University Global Business Outlook Survey of 1,275 finance executives, the most pressing concern for finance professionals is inaccurate forecasting. Crowdcast has helped media, technology, manufacturing and health care companies to improve forecast accuracy by as much as 60 percent, providing actionable data that can save those firms millions of dollars.

Typical Crowdcast Prediction Market Results:
• Media Firm – 1,200 employees predict quality, sales, and ship dates, resulting in61% less error versus management
• Pharmaceutical Company – panel of scientists and doctors predict regulatory decision sand new drug sales; more accurate 86% of the time

“Employees know what’s really going on in a corporation, but that information rarely reaches management’s ears. By tapping employee intelligence, Crowdcast channels this raw knowledge into actionable business intelligence,” said Mat Fogarty, Crowdcast Founder and CEO. “Crowdcast is the first enterprise-ready prediction markets platform that gives executives real-time data about progress toward specific goals so they can take action.”

An effective corporate prediction market offers users a simple interface, requires minimal interaction and allows easy communication of rich information. Crowdcast’s platform is the first prediction market designed specifically for the corporation, utilizing a cutting edge market design that moves far beyond the legacy stock-based trading systems. With Crowdcast, employees place bets on various corporate outcomes, compete with each other, and are rewarded for their accuracy.

“The Crowdcast platform leverages novel algorithms that are grounded in economic theory and experimentation, yet it is usable and engaging. The result is a best-in-class solution for enterprise crowd forecasting,” says Peter Coles, Assistant Professor at Harvard Business School. Crowdcast recently received Series A funding from Alsop Louie Partners.

http://www.crowdcast.com/

UPDATE: Mat’s post

- CrowdCast on LinkedIn

- CrowdCast on FaceBook

- CrowdCast on Twitter

- VentureBeat

Next: In most cases, the team (including the best predictors) outshines the best predictor working alone.

CrowdCast’s Leslie Fine:

The claim here is not that the administrator knows more than the executive. The claim is that the executive’s point-of-view can be enhanced by mingling it with the admin’s. Empirical and experimental evidence show that in most cases the team (including the best predictors) outshines the best predictor working alone. Furthermore, in our implementations we find that our front line managers often have earlier information than the executive suite. Our mission is not only to provide accurate forecasts, but to do so as early as possible.

Andrew McAfee on CrowdCast

UPDATE:

Leslie Fine (CrowdCast Chief Scientist) to me:

UPDATE: Wall Street Journal

Can prediction markets help improve economic forecasts?

Michael Giberson — Mon, 15 Jun 2009 16:40:57 +0000

At VOX, David Hendry and James Reade examine the question, “How should we make economic forecasts?” Among the ideas discussed is whether prediction markets could be used to improve economic forecasting. Interesting suggestion and seeming to be worthy of additional exploration, but the authors don’t go too deep here. Instead, they assert that “prediction markets can be viewed as a form of … model averaging,” and then drift into a discussion of forecast averaging. I’m not sure that forecast averaging is a good way to look at prediction markets.

Here is what they say:

Prediction markets can be viewed as a form of forecast pooling or model averaging, a common forecast technique (Bates and Granger 1969, Hoeting et al 1999 and Stock and Watson 2004). That is, forecasts from different models are combined to produce a single forecast. In prediction markets, each market participant makes a forecast based on his or her own forecasting model, and the market price is some function of each of these individual forecasts.

Since the “prediction” implied by a prediction market is set by the marginal transaction, it depends not at all on the distribution of earlier trades, nor on the valuations of parties priced out of the market at the current price.

For example, consider two event markets: in the first 999 contracts trade at $0.50 and the 1000th and final trade is at $0.75; in the second 999 contracts trade at $0.76 and the 1000th and final trade is at $0.75. In the typical interpretation of prediction markets, the event is “predicted” to result with a 75 percent probability in both cases. However, averaging among the different predictions doesn’t get you that result.

(Well, strictly speaking the market price is “some function” of the prices – namely, one in which all trades but the last are weighted zero and the last trade is weighted one. You can call this “averaging,” but that isn’t the most useful explanation of the function.)

I’m not arguing that forecast averaging might not be a good idea in many situations, just that averaging doesn’t seem like a good way to explain what a prediction market is doing.

MicroSoft have patented the prediction markets.

Chris F. Masse — Tue, 26 May 2009 17:30:06 +0000

Well, at least, one aspect of the prediction markets.

Uniited States Patent Application:

CONTINUOUS BETTING INTERFACE TO PREDICTION MARKET

Abstract

A user participates in trading securities in a prediction market which represent different outcomes of an event, using an interface which allows the user to understand a trade in terms of a bet. The interface also allows the user to explore different bet amounts while receiving feedback on potential payoffs in real time. In one approach, a tool is used which is moved in one direction by the user to indicate a larger trade for an outcome, or in the opposite direction to indicate a larger trade against the outcome. A bet for an outcome can be translated into a purchase of a corresponding security which represents the outcome. A bet against a particular outcome can be translated into a purchase of securities which represents all other outcomes. The user interfaces hides complexities of the market from the user while surfacing relevant information.

Inventors: BERG; HENRY G.; (Redmond, WA) ; PROEBSTING; TODD A.; (Redmond, WA)

I won’t comment on this, because I believe that the US patent system is broken, generally speaking.

The Fox… and the Hedgehog

Chris F. Masse — Thu, 07 May 2009 20:11:27 +0000

Emile Servan-Schreiber looks like the fox to me. He acknowledges a reality: the prediction market adoption problematic. (Leslie Fine is in the same boat.) Okay, but I want more proof. How come Inkling Markets administrators seem to be happy with MSR, then?

Adam Siegel is no hedgehog.

Addendum

Here is the definition of a fox (as opposed to a hedgehog):

a thinker who knows many little things, draws from an eclectic array of traditions, and is better able to improvise in response to changing events;
a self-critical, eclectic thinker who is willing to update his/her beliefs when faced with contrary evidence, who is doubtful of grand schemes and is rather modest about his/her predictive ability.

Why CrowdCast ditched Robin Hanson’s MSR as the engine of its IAM software

Chris F. Masse — Wed, 29 Apr 2009 17:19:39 +0000

Leslie Fine of CrowdCast:

Chris,

As Emile points out, in 2003 I started experimenting with (and empirically validating) alternatives to the traditional stock-market metaphor that will be more viable in corporate settings. We found the level of confusion and lack of interest in the usual fare led to a death spiral of disuse and inaccuracy. BRAIN was a first stake in the ground in prediction market mechanism design with usability as a fundamental premise.

When I joined Crowdcast (then Xpree) in August of 2008, Mat and the team already recognized the confusion around, and consequent poor adoption of, the MSR mechanism. The number of messages I fielded in my first month here asking me to explain pricing, shorting, how to make money, etc. was astounding. We all knew that we had to start from scratch, and rebuild a mechanism that was easy to use, expressive both in terms of the question one can ask and the message space in which one can answer, and provided a high level of user engagement. We have abandoned the MSR in favor of a new method that users are already finding much simpler and that requires a lower level of participation and sophistication than the usual stock market analogy.

I wish I could go into more detail. However, we need to keep a little bit of a lid on things for our upcoming launch. I can only beg your patience a little while longer, and I hope you will judge our offering worth the wait.

Regards,
Leslie

Nota Bene: IAM = information aggregation mechanism

UPDATE: They are out with their new collective forecasting mechanism.

Simpler Input Mechanisms

Chris F. Masse — Tue, 28 Apr 2009 17:43:19 +0000

I really like what Emile Servan-Schreiber replied to Robin Hanson. I really like it. However, if I am correct NewsFutures never implemented MSR, so how can Emile be so affirmative? Rather, I would like to hear from Mat Fogarty or Leslie Fine (of Xpree/CrowdCast) who did implement MSR, and recently decided to move out to non-trading mechanisms.

UPDATE: The Fox… and the Hedgehog