I will soon publish a full post on the relative accuracy of the VP prediction markets (and whether the prediction markets are useful at all, taking into account the occasional upsets theorized by Koleman Strumpf) —-tackling Paul Kedrosky, Felix Salmon, and Barry Ritholtz.
- By the way, I was pleased to see (elsewhere) that Emile Servan-Schreiber of NewsFutures has the same (bad) opinion about those VP prediction markets as I do. I will index Emile in this file, next week. Emile is a smart and experienced man, and we have the confirmation of this under our very nose (elsewhere), once again. Emile Servan-Schreiber = one of the great thinkers of the field of prediction markets.
- I was also pleased to see that our good Wall Street friend Eddy Elfenbein got things right.
Now, back to the PM-bashing Paul Kedrosky post:
Paul Kedrosky criticizes the “-boosters”- of the prediction markets —-Justin Wolfers, Robin Hanson, John Delaney, Chris Masse, etc.
As for me, I dislike opportunistic bloggers and venture capitalists like Paul Kedrosky who approach the prediction markets without a basic understanding of the forecasting approach.
Here is the explainer that I have been publishing on the frontpage of Midas Oracle, for all to see.
Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that traders bring when they agree on prices. Prediction markets are meta forecasting tools that feed on the advanced indicators (i.e., the primary sources of information). Garbage in, garbage out…- Intelligence in, intelligence out…-
A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative can be interpreted as the objective probability of the future outcome (i.e., its most statistically accurate forecast). A 60% probability means that, in a series of events each with a 60% probability, then 6 times out of 10, the favored outcome will occur- and 4 times out of 10, the unfavored outcome will occur.
Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism.
Finally, for those who missed it, the great interview given by Koleman Strumpf to CNN, in April 2008.
FOREMAN: I’-ve got something I want you to take a look at. Look at this. It could be the price of a stock or a mutual fund. It isn’-t. It’-s the odds that a particular candidate, the red here is Hillary Clinton, who will become president of the United States. It’-s called the [prediction] market. And while supporters say it’-s no different than any other type of investment, for example, hog bellies or pork futures, or crude oil, it sure looks like gambling.
Economics Professor Koleman Strumpf is on the faculty at the University of Kansas School of Business to talk about it now. And in Las Vegas, another scholar in the art of predicting future events, Johnny Avello, the director of the Sports Book at Wynn, Las Vegas.
Let me start with you, professor. How well does this work? When people start betting online as to who’-s going to win, does it work?
KOLEMAN STRUMPF, UNIVERSITY OF KANSAS: Yes, the markets have a really tremendous track record dating back at least on the online markets to 1988. And actually earlier, there were impromptu markets that existed outside of Wall Street in the early 20th century.
These things have been around for at least 100 years or probably more. 150 years. And with maybe only one or two exceptions that I can think of, they’-ve done a just totally dead on job at forecasting.
[FOREMAN]: Long before polling existed, then you’-re saying we had betting. And there were betting lines in newspapers. And if you want to know who’-s winning the presidential race, that’-s what you looked at?
STRUMPF: That’-s exactly right. So in 1904, “-The New York Times”- reported on the front page what was going on at the Wall Street betting markets since there was no Gallup Poll that existed at that time.
FOREMAN: Johnny, why do you think that this is generally so successful compared to polling?
JOHN AVELLO, DIR. OF RACE &- SPORTS, WYNN HOTEL: Well, first of all, you are taking actual bets. And you know, each person that puts their money up is a good indication of, you know, which way they like it.
When you’-re doing polling, you know, that’-s kind of an ambiguous way of finding out who the winner is because you’-re getting a fraction of the people who you’-re actually finding out who they like. So I like to call it money versus unpredictability.
FOREMAN: You’-re saying the difference is that in a poll, somebody may say something that they believe in generally, or they think that the pollster wants to hear. But when they put money down, they’-re going to really bet on what they think is going to happen?
AVELLO: It’-s the real thing.
FOREMAN: It’-s sort of interesting when you look at these different types of polls. There’-s the In Trade system, which is out of Dublin, Ireland that’-s sort of interesting. In Trade allows people to bet actual money and large amounts of it on all sorts of outcomes all around the world.
The Iowa markets is one the people have talked about a lot. Set up by a university there. Basically the Iowa markets allow people to bet in limited amounts of money. About $500. And it’-s used simply to see how well this works for educational purposes mainly.
Professor, when we talk about these, though, why did polling ever become popular if betting works so well in telling how we would win?
STRUMPF: Well, I think a kind of —- much like today, the newspapers, the media was always uncomfortable reporting these markets. There sort of was dubious legal status and maybe some moral issues with it.
Polling for whatever reason seems to have been morally acceptable to the media. And I think as a result when Gallup came around in the late 1930s, the betting markets kind of fell by the wayside. They never of course disappeared, but I think it was sort of a moral issue, the same kind of moral issues that I think arise today in thinking about gambling.
FOREMAN: So let me ask you this, Johnny. Why do you think it’-s been so difficult this year, though? Because as far as we can tell, the betting lines have not done any better than the pollsters this year in predicting this election. It has been all over the map. And the betting has been all over the map.
AVELLO: Well, one thing to remember, Tom, is that when the book maker’-s putt
ing up a line, what they’-re trying to accomplish is divided action. So they’-re not trying to pick the winner because let’-s take for instance if the book maker put up Hillary Clinton at one to two, and she was bet from to three to win the Democratic nomination. You know, and Obama won. People would say, wow, the book maker really got killed on that.
FOREMAN: The book maker’-s not trying to predict it, but obviously, the gamblers are, the people who are betting on these things. And they haven’-t done well this time, not compared to past elections. Why do you think that is? Why is this so hard to sort out?
AVELLO: Because they’-re not always right. No one’-s right at 100 percent of the time. I would say best case. You know, I know that history has shown that the bettor has done well on this. But to be perfectly honest with you, I think if you do 60 percent, you’-ve done a great job of picking the winner.
FOREMAN: And Koleman, do you think there’-s anything unique that’-s making it harder for the betting markets to be as accurate as they had been in the past?
STRUMPF: Well, it’-s obviously a close market and opinions are changing rapidly. I just want to kind of maybe extend a little on what Johnny just said in terms of thinking about what these markets mean.
The markets give us a probability of an event occurring. So even if, for example, Obama is a 80 percent favorite in the upcoming Wisconsin primary, which he is, that still means on the flip side that there’-s a 20 percent chance that he’-s not going to win. So the markets, again as Johnny had said, don’-t – they – by sort of definition can’-t predict an upset. An upset is a surprise which people hadn’-t anticipated. So sometimes there are these quick shifts of opinion, which I —- to the best of my knowledge, there’-s no way to forecast that in advance.
FOREMAN: And this campaign has been just filled with them. Johnny, one last thing here. Any sense of where the smart money is going these days?
AVELLO: Well, let’-s say that there’-s been a shift. I believe the smart money was on Hillary Clinton early, and has shifted to Obama. But surprises do happen. And all you need to do is look at the Superbowl to find that out.
FOREMAN: Johnny, thanks so much. Koleman, as well. We appreciate you being here. And speaking of gambling, Madame Tussaud’-s Wax Museum here in Washington is hedging its bet. And that kicks off our political side show.
Back to the PM-bashing Paul Kedrosky post.
Here’-s the sneaky, un-informed and totally biased ( ) comment from our good friend Jason Trost (who should know better), which was attached to the Paul Kedrosky post:
UPDATE: Portfolio + Computational Complexity
I’-m free to talk, now.
The Industry Standard is powered by Consensus Point.
The New York Times don’-t print that, but they print that MIT CCI’-s Thomas Malone (branded in the piece as a prediction market evangelizer) has been advising The Industry Standard.
I spotted dozens of news articles on the Industry Standard’-s re-launching. Their spin doctor did a good job.
By the way, speaking of media-managed prediction exchanges, the CNN prediction exchange has some prediction markets with each a total of transactions in the magnitude of 50,000. That’-s awesome. Congrats to Inkling Markets. Mike Giberson (who has become an expert in MSR trading) is one of the traders, probably…-
The goal of CNN Political Market is to combine the opinions of a diverse group of people to try and predict the probability of an event occurring or the value of something. Why is this important? Because more often than not, a diverse group of people or “-crowd”- will generate a more accurate prediction than an individual or a small group of “-like-minded”- or “-single-discipline”- folks.
In business, politics, and culture, this can have big ramifications:
– Predictions often turn out to be more accurate than surveys and polls-
– More accurate forecasts affect how marketing dollars are spent, how many widgets should be built in the first run, etc.-
– Decision making is more democratized, giving everyone input where they may not have had it before-
– Markets can serve as on-going indicators for key performance metrics.
Frequently Asked Questions
These are answers to questions we receive on a regular basis.
Q: What if I forgot my username or password?
A: It’-s easy, just click here and new information will be sent to the email address we have for you.
Markets and Trading
Q: How do the prices in a market change?
A: Markets are guided by the forces of supply and demand. Every time a user buys a share in a particular idea or outcome, that demand forces the price up a little. The more people buy, the higher the price becomes. If the price gets too high, people will do the natural, self-interested thing and sell their shares to get a profit (or sell shares that they don’-t own on credit). Similarly, each time a user sells a share, the price goes back down some.
Q: My math says I can buy more shares than the system lets me. Why?
A: Remember that for each share you buy, the price goes up some. If you’-re buying a lot of shares at once, the last share in your order might cost significantly more than the first.
Q: I sold a block of shares, but I got less money out of it than I thought I would.
A: Just like the market price increases you see when you purchase shares, each share you sell drives the market price for those shares down a little. If you’-re selling a lot of shares, the last share you sell be worth much less than the first.
Q: I think the current price in a market is reflective of what the market is trying to predict. What do I do now?
A: The best thing to do is to hold onto your shares. If you try to sell them, the market price will start to go down and you won’-t make as much profit as you could if you wait for the market to close.
Q: What is “-on credit?”-
A: When you sell shares on credit you are betting that the market price of a stock is going to go down instead of up. The idea is to sell shares you don’-t have at a high price, and buy them back later once the market price has fallen. You get to keep the difference as profit.
Because you never know how high a price will go up, we’-ve set up specific rules for selling shares on credit so you don’-t get yourself in to trouble:
– You can buy back shares you bought on credit with the money in your bank.
– There is a limit on how much stock you can sell on credit. You can figure out what that limit is by adding your available balance to the total value of shares you have bought. For example, if you have $10 available balance, and you also own $10 of stock, you can sell $20 of stock on credit. In other words, we try not to allow you to go in to debt by always protecting you against the worst possible scenario.
Q: How can I tell how much money I have?
A: Your balances are listed in the “-total assets”- box which you can find on your dashboard/portfolio. Your balance is also listed on each market trade page.
Q: How can I get more money?
A: You can earn money by trading wisely in markets. Look for good ideas that are undervalued, examine market descriptions for interesting things that others might not have noticed, or use your own unique, personal knowledge to make predictions that others can’-t.
Q: What is the play money good for?
A: Status in the online trading community is based on how much money you have- just take a look at the “-top traders.”- The more money you have, the sooner everyone will begin to refer to you as “-sage.”-
Q: Can I give some of my money to another user?
A: Money is non-transferable – you must earn your own by trading.
Marketplace and Help
Q: The website looks a little funny. Is my computer supported?
A: We support the latest versions of Internet Explorer, Firefox, and Mozilla for Windows XP and 2000. For Mac OS X we support the latest versions of Safari, Firefox, and Mozilla. The application may work in alternate OSes and browsers, but we can’-t make any promises.
As Chris alluded to a few days ago in a post, we’-ve worked with CNN to launch a marketplace for this election season. A smattering of markets are available now with more to come, I’-m told. So if you like trading in Inkling and want to participate in what I assume will quickly become our largest marketplace (it’-s featured now on http://cnn.com and the inbound traffic is “-remarkable”- to say the least) you can go here: politicalmarket.cnn.com
American politics does not suffer from a shortage of polls. Zogby. Gallup. Rasmussen. SurveyUSA. Mason-Dixon. Polimetrix…- In an information-glutted world, what matters is not the supply of sources, but the ability to glean trustworthy information from the larger swath of poor data.
Different polling organizations have different strengths and weaknesses. Some use “-tight screens”- to scope out likely voters- others simply sample registered voters, without making any attempt to tighten the survey base to “-likely voters.”- Tight screening is especially crucial to gauge the true state of a primary, when committed base opinion can diverge significantly from less engaged moderate voters, and more importantly, influence those moderates over time to converge to the more partisan perspective. Some use human interviewers, although recently that has given way to IVR (Interactive Voice Recording) polls (the kind where a computer talks to you and asks you to “-press 1 if you will definitely support X, 2 if probably…-”-)
I have found tight-screen, IVR polling to be the most reliable. IVR not only has no marginal cost, but it eliminates all the biases resulting from trying to give the most pleasant-sounding answer possible (the “-sexy grad student effect”- that exaggerated Kerry’-s margin by 15 points in Pennsylvania 2004 exit polling, for example). IVR possible responses can also be randomly rotated from respondent to respondent to eliminate recency biases (first and last responses in a list exaggerated because those are at the forefront of a person’-s memory of the list, not because s/he will vote that way).
The poster-child of IVR tight-screen polling success is Scott Rasmussen’-s Rasmussen Reports. I have only tracked them over the last two election cycles (2004 and 2006), but considering that 2004 was a GOP wave and 2006 a Democratic wave election, I think the data is sufficient to form a valid judgment. Rasmussen’-s track record is simply stupendous. It predicted 49 out of 50 states in 2004 correctly, usually within two percentage points of the actual outcome. In 2006, Rasmussen achieved similarly impressive results —- all the more impressive when you consider that most polling models tend to err in favor of one party or the other. (“-Likely voter”- models tend to favor Republicans, and registered voter-based models tend to exaggerate Democratic strength.)
My other favorite sources include Gallup and Mason-Dixon. Gallup comes closer to the “-registered voter”- model than the tighter Rasmussen model, so Gallup usually lags tighter-screen polls. By election eve, however, the two models usually converge. Gallup’-s election-eve congressional generic vote is hands-down the best in the business. However, their numbers for party primaries have poor predictive value, because they don’-t make much effort to hunt down likely voters.
Differing survey methods can yield very different results. Rasmussen has long shown a much closer Democratic nomination race than most established, “-registered voter”- pollsters —- most recently, it showed a 32-32 tie between Clinton and Obama, with Edwards wallowing 15 points behind. Gallup’-s last numbers tightened drastically to a 31-26 race between Clinton and Obama (Gallup’-s numbers are also hard to compare with Rasmussen’-s because Gallup includes Gore).
Many smart Democrats, notably MyDD’-s Chris Bowers, believe that Gallup and others are mistakenly including lots of “-low information voters”- who simply lag the opinions and thought processes of more-attuned Democratic partisans.
Now that more establishmentarian polling firms are coming in line with Rasmussen’-s results, one can infer that the likely voter/ Chris Bowers theory has gotten the better of the argument.
A survey of pollsters wouldn’-t be complete without knowing which ones to stay away from. Stay away from Zogby and CNN polling. James Carville’-s and Stan Greenberg’-s DemocracyCorps polling outfit is not trustworthy, either —- for example, when they doubled the percentage of blacks in an October 2006 survey sample to bump the Democrats’- generic advantage by 5 points, to reinforce the Democratic narrative of a building wave.
Lastly, partisan pollsters in a competitive election season should always be taken with a grain of salt —- they will use heuristic subtleties to create the best impression possible for their party’-s candidates. Strategic Vision, a Republican outfit, deserves a three- or four-point handicap. Franklin Pierce generated a dubious Romney result for New Hampshire right after its lead pollster, Rich Killion, went to work for the Romney campaign. Such polls should be trusted only as a last resort.
For those of us who wish to divine movements in politics futures, discerning trustworthy data from bad data is paramount. Poll-rigging is the high art of Washington, DC, and as any interest group —- or candidate —- knows, it’-s easier than easy to produce a poll that diverges wildly from reality, if the heuristics are threatening enough.