Category Archives: Accuracy (Absolute Relative) Precision

How accurate are prediction markets in US elections?

Posted on by
Reply

No GravatarUnless you&#8217-re just just surfacing from an Afghan cave, you can easily guess who that &#8220-David&#8221- could be&#8230- :-D

–

Yahoo! Answers on prediction markets

Previous blog posts by Chris F. Masse:

  • The Most Surprising Piece Of News I’ve Heard Today
  • My first prediction market plugin for WordPress
  • Self-Serving Prediction Market Of The Day — Unlawful Internet Gambling Enforcement Act of 2006
  • Prediction markets tend to be so illiquid, though, that mere activity looks like volatility.
  • Decision Markets and Futarchy are solutions in desperate search for a problem to solve and for their early adopters… and that may stay that way well after Robin Hanson’s head gets cryogenized.

OSCARS 2008: The Hollywood Stock Exchange has been more accurate than InTrade.

Posted on by
4

No Gravatar

–

&#8230- dixit James Surowiecki (commenting on Felix Salmon&#8217-s post):

Interesting. The Hollywood Stock Exchange, not surprisingly, did better [than InTrade]. Cotillard was, as at Intrade, a comfortable second favorite. But so too was Swinton &#8212- in fact, she was even more of a second favorite, as her price was very close to Blanchett&#8217-s when the market closed. In fact, if you look at her chart:
http://movies.hsx.com/servlet/SecurityDetail?symbol=A8TSW

it&#8217-s hard not to conclude that the market was really incorporating new information in the week leading up to the ceremony.

I didn&#8217-t follow the Hollywood Stock Exchange [*] (or even BetFair) closely for the Oscars 2008, but here are InTrade&#8217-s expired event derivatives (event futures):

Best Picture

Best Director

Best Actor

Best Actress

–

[*] UPDATE: HSX claims a 75% success rate.

–

Prediction markets produce dynamic, objective probabilistic predictions on the outcomes of future events by aggregating disparate pieces of information that traders bring when they agree on prices. Prediction markets are meta forecasting tools that feed on the advanced indicators (i.e., the primary sources of information). Garbage in, garbage out&#8230- Intelligence in, intelligence out&#8230-

A prediction market is a market for a contract that yields payments based on the outcome of a partially uncertain future event, such as an election. A contract pays $100 only if candidate X wins the election, and $0 otherwise. When the market price of an X contract is $60, the prediction market believes that candidate X has a 60% chance of winning the election. The price of this event derivative can be interpreted as the objective probability of the future outcome (i.e., its most statistically accurate forecast). A 60% probability means that, in a series of events each with a 60% probability, then 6 times out of 10, the favored outcome will occur- and 4 times out of 10, the unfavored outcome will occur.

Each prediction exchange organizes its own set of real-money and/or play-money markets, using either a CDA or a MSR mechanism.

–

Intrade, with carry

Posted on by
2

No Gravatar

For years now, a criticism of Intrade has been its lack of a positive carry in long-term markets. In contrast to regulated futures exchanges, Intrade does not pay out interest on deposits to most traders. While understandable, this discourages liquidity and can skew prices near extremes as future winnings are discounted, exaggerating any favorite/longshot bias.

If carry can be seen as a type of liquidity subsidy, it need not be provided by the exchange. It could be paid by other traders, or passed-on by traders that happen to have interest-bearing accounts. Consider a contract that is set to expire at 100 with certainty at a future date. Offers below 100 then represent a carry payment, and prices in such a market imply something like a discount rate. This is not a true interest payment or discount rate because no funds are being loaned, but in some situations the buyer will gain the benefit of reduced margins on subsequent trades in addition to &#8220-interest&#8221-.

Take the new &#8220-tax futures&#8221-, which are structured as a &#8220-ladder&#8221- of binary options. Currently the lowest rung expires at 100 if the highest marginal tax rate is equal to or greater than 34% and zero if it is less than 34%. The next highest rung pays when the highest marginal rate &gt-= 36%, and so on. Say we add a new rung that pays-off if the rate is greater than 0% and has no transaction fees. This would seem to be a relative certainty, and an offer below 100 represents carry. Furthermore, the buyer of this contract can then sell short higher rungs at reduced margin.

For example, with the 2011 maturity contracts, the current bid/ask on the &#8220-&gt-38&#8243- rung is 85/87.5. If a &#8220-&gt-0&#8243- rung existed and someone sold it for 95, the buyer can then go out and short &#8220-&gt-38%&#8221- at 85 with 10 margin (corresponding to his worst-case scenario of negative tax rates: 85-95!) instead of the usual 15 margin frozen, plus collect the carry of 5 in the &#8220-&gt-0&#8243- contract by maturity. In absolute terms, the margin is only reduced by the unannualized carry of 5, but if the trader is right, he will capture a return on frozen margin of 900% ((85+5)/10-1) instead of 467% (85/15-1).

Likewise, an &#8220-impossible&#8221- contract would subsidize buyers. The 2009 contracts are currently 35 ask for &#8220-&gt-38&#8243-. If a &#8220-&gt-100&#8243- rung existed and someone bought it for 2, the seller could then buy the &#8220-&gt-38&#8243- with 33 margin frozen – a subsidized 203% vs. a regular 186% profit in this case. (The &#8220-sponsor&#8221- is not interested in profiting directly on that trade and so the fact that the government would be owed all winnings if they are a US citizen doesn&#8217-t matter.)

Now it&#8217-s fair to ask why you couldn&#8217-t accomplish the same subsidy by just consistently selling for slightly less than and buying for slightly more than what you judge to be fair value. Aside from the facts that this could widen the bid/ask spread and make prices less reliable, the more explicit carry of the system described above is psychologically reassuring and should lead to more trade. It is very hard to distinguish between a 65% and a 67% chance of the next president being a Democrat – for good reason. How can a trader practically determine if a 2% difference in price is a subsidy? At the same time, the sponsor&#8217-s risk is more defined if they stick to contracts with near-certain payoffs. In many cases the sponsor will also be a market-maker, but this system allows those functions to be decoupled.

It could be argued that the risk-profile of the average Intrade trader is such that they are insensitive to these kinds of incentives because they are generally trying to grow small sums of money by orders of magnitude, as opposed to preserving larger sums. Even if this is an issue, the multiplier effect described above should mitigate it.

A more likely problem with this system is the danger of free-riders taking carry without providing liquidity to the sponsored market. The sponsor could deal with this by only extending small subsidies at any one time and ceasing to do so if he doesn&#8217-t see reciprocation. The fact that frozen margin on the trade earning carry is unfrozen only when an offsetting trade is made on the same ladder also discourages free-riders. The system could somewhat easily unravel though, with the &#8220-discount rate&#8221- collapsing to relatively useless levels. This would not be surprising, especially in the &#8220-&gt-100&#8243- contract, although subsidizing tax short-sellers with the &#8220-&gt-0&#8243- contract is more important in this market.

Remember, this entire idea is meant to work within the technical and legal structures as they exist.

Felix Salmon slams Wall Street Journals Justin Wolfers -but not Chris Masse.

Posted on by
Reply

No Gravatar

Felix Salmon:

[…] And thirdly, if you&#8217-re Justin Wolfers, it&#8217-s probably smart not to make unhedged statements saying that Barack Obama has &#8220-better than a nine-in-ten chance of winning&#8221- the New Hampshire primary. […]

Previously: Prediction markets are forecasting tools of convenience that feed on advanced indicators.

Prediction markets are forecasting tools of convenience that feed on advanced indicators.

Posted on by
2

No Gravatar

Why were the political prediction markets so wrong about Barak Obama and Hillary Clinton in New Hampshire?

&#8230-asks Slate&#8217-s Daniel Gross &#8212-via Mister Usability (Alex Kirtland), who needs to go and get his own gravatar.

So, I&#8217-ve been watching the action in one of the political futures markets this evening, Intrade. And the action in this prediction market has reinforced my opinion that these are less futures markets than immediate-past markets. The price movement tends to respond to conventional wisdom and polling data- it doesn&#8217-t lead them.

Throughout the day and into the early evening, while polls were still open, Democratic investors, mimicking the post-Iowa c.w. and polls, believed Obama was highly likely to be the Democratic nominee. The Obama contract was trading in the lows 70s, meaning investors believed he had a 70 percent chance of being the nominee, while Hillary Clinton contracts were in the 20s. […] At 6 p.m., this market had written Hillary Clinton&#8217-s entire presidential campaign off. At 9:30 p.m., it was calling a dead heat. What caused investors to change their minds so drastically in the space of a couple of hours? A few data points that went against the day&#8217-s prevailing conventional wisdom and polls. […]

See also Niall O&#8217-Connor&#8217-s assessment:

I am looking forward to the post New Hampshire Caucus, when all you prediction market advocates crawl out from under your stones. For the record at one point the market on Intrade and Betfair was suggesting that Obama had a 95% probability of winning the caucas- whilst Intrade had him at 77% to win the nomination.A case perhaps of both the foolery of crowds and, the market biting back.

New Hampshire will go down as the Black Wednesday of prediction markets and unless there is now objective transparent debate (as opposed to the usual biased sabre rattling) – prediction markets will be dead in the water.

My answer to Dan Gross&#8217- legitimate question and to Niall O&#8217-Connor&#8217-s snarky comment:

  1. Prediction markets are forecasting tools of convenience that feed on advanced indicators. When those advanced indicators are wrong, the prediction markets are wrong.
  2. If you prefer the polls or the pundits, your call &#8212-but polls and pundits were also wrong, this time, right? Required reading for mister Niall O&#8217-Connor: &#8220-New Hampshire&#8217-s Polling Fiasco&#8221- + &#8220-Analysis: pundits eat crow&#8220-.
  3. The ultimate forecasting tool would be a way to reverse our psychological arrow of time &#8212-so as to remember the future instead of the past. Only science-fiction writers and some imbecile ( :-D ) believe in that.
  4. The prediction market approach is to stick with the markets, on the long term. Take their successes. Take their failures. Unlike Donald Luskin and Markos Moulitsas, Chris Masse will not turn against the prediction markets when they fail punctually. What counts is the long series.
  5. My first point should be included in the prediction markets approach definition, in my view, but others (like economist Michael Giberson) might have different opinions.
  6. With respect to my first point, I bet that the prediction markets will never replace the polls as the forecasting tool of choice for political analysts &#8212-on that particular point (but not on a myriad of others), I break away from Justin Wolfers&#8217- irrational exuberance and I side with Emile Servan-Schreiber of NewsFutures (my preferred play-money prediction exchange). Prediction market reporting will have a function, indeed (as suggests Justin Wolfers), but not the dominant function.
  7. Going forward, prediction market journalism should emphasize relative accuracy (as opposed to absolute accuracy) &#8212-that is, comparing prediction markets with polls and pundits, which is what Robin Hanson has said from day one. Our good friend Niall O&#8217-Connor has difficulty to compute that, apparently. He should eat more fish. :-D

&#8212-

Justin Wolfers:

“In a few years, we may regard the second half of the 20th century as the aberration in which the press used polls rather than markets to track political races,” Justin Wolfers, a business professor at the University of Pennsylvania’s Wharton School, wrote in an e-mail message. “And in the 21st century, we may return to the habits of the early 20th century, reporting on political races through the lens of prediction markets rather than polls.”

Emile Servan-Scheiber:

1) The traders themselves are the first to look at the polls to inform their trades. So the polls are here to stay.

2) Our recent experience in Western Europe seems to indicate that the superior accuracy of markets over polls when predicting elections may be a U.S. artifact that isn’t so easily reproducible elsewhere. I’ve discussed this with Forrest Nelson of IEM [Iowa Electronic Markets], and apparently, ever since the Truman-Dewey polling debacle of 1948, U.S. pollsters have adopted a policy of reporting mostly raw numbers rather than projections based on sophisticated secret formulas, so they can’t be accused of manipulating opinion. However, raw numbers are notoriously unreliable when based on small samples, and Western European pollsters never report them, preferring instead to publish projections based on historically-informed statistical formulas. What we’ve observed in France and Holland is that it it’s very hard to beat the accuracy of such projections.

[I don’t make mine Emile Servan-Schreiber’s second point, but that’s a minor.]

&#8212-

InTrade&#8217-s expired prediction markets:

&#8212-

New Hampshire

&#8212-

The Democrats

&#8212-

The Hillary Clinton event derivative was expired to 100.

Dem NH Clinton

Dem NH Obama

Dem NH Edwards

&#8212-

The Republicans

&#8212-

The John McCain event derivative was expired to 100.

Rep NH McCain

Rep NH Romney

Rep NH Huckabee

Rep NH Giuliani

&#8212-

Iowa

&#8212-

The Democrats.

The Barack Obama event derivative was expired to 100.

Dem Iowa Obama

Dem Iowa Clinton

Dem Iowa Edwards

&#8212-

The Republicans

The Mike Huckabee event derivative was expired to 100.

Rep Iowa Huckabee

Rep Iowa omney

Rep Iowa McCain

&#8212-

Source: InTrade

&#8212-

[A more complete prediction market reporting should have included expired contracts from NewsFutures and BetFair. Sorry for that. Note that InTrade-TradeSports is the only exchange to offer a “closed contacts” section.]

&#8212-

NEXT: Prediction Markets 101 + Who did best in explaining the prediction markets to the lynching crowd? + After the New Hampshire fiasco, 16 people came to defend the prediction markets, so far. + The prediction markets deserve a fair trial. + Prediction Markets = the greatest time-saving invention of this century

&#8212-

Polls over prediction markets?

Posted on by
Reply

No Gravatar

&#8216-Are Political Markets Really Superior to Polls as Election Predictors&#8217- is a paper by Robert Erikson and Christopher Wlezian that calls into question whether prediction markets, and specifically IEM, is as accurate as proponents of prediction markets claim. The paper was highlighted here where it was suggested that advocates of prediction markets were turning a blind eye to it.

In the paper the authors show that while when you compare IEM with raw polls, IEM outperforms the polls however when you manipulate the poll data the polls are more accurate. The generate the manipulation the authors looked at poll data from elections from 1952 onwards which show that over time the early leader tends to lose that lead. They then used that relationship to manipulate polls for elections from 1988 onwards and compared the result with the IEM forecasts. The manipulated polls showed a higher level of forecast accuracy.

I think this is an interesting piece of research but it is a stretch to use this to claim that polls are more accurate than prediction markets. The fundamental problem is that when newspapers (or anyone else for that matter) quotes polls, they don&#8217-t refine them using historical data, they quote the actual poll result. If anything the authors have shown a small bias in IEM that one would now expect to get traded out (like the longshot/favorite bias in betting markets or the January effect in financial markets).

Fundamentally, the thing to note is that while polls make prediction markets more accurate, the converse does not hold.

Scarce liquidity in the financial prediction markets run by InTrade-TradeSports

Posted on by
6

No Gravatar

The Daily DOW prediction markets are thin, and the Daily Nasdaq prediction markets are inexistent.

Here is the data for yesterday.

DOW:

DOW Oct 29, 2007

Nasdaq:

Nasdaq Oct 29, 2007

Previous blog posts by Chris F. Masse:

  • The marketing association between BetFair and TOTE Tasmania works better than expected.
  • The term “event markets” sucks —and the uncritical thinkers using this crappy term suck too.
  • CLIMBING HIS WAY TO THE TOP: Erik Snowberg is now Assistant Professor of Economics and Political Science at California Institute of Technology.
  • Unlike other countries, the United States of America defends the freedom to offend in speech.
  • The best research papers on prediction markets
  • 2008 Electoral Map
  • American Enterprise Institute’s Center For Regulatory And Market Studies (Policy Markets)

Does this prediction market chart look predictive to you?

Posted on by
4

No Gravatar

Karl Rove will resign from the White House.

Karl Rove resignation - NewsFutures

Emile Servan-Schreiber:

Chris, how exactly do you define “predictive”? If your criterion is “last trading price above 50%”, that would betray a very limited understanding of the nature of both probability and binary markets. That’s a debate you and I have had ever since the first days of chrisfmasse.com a propos the 2004 US presidential election.

To your credit, I don’t think anyone has yet proposed a good way of assessing the “predictiveness” (predictivity?) of a single binary market after the fact. It is a very difficult question. Does anyone here have an answer?

#1. What I see on the NewsFutures chart above is that the probability of Karl Rove resigning went to about 20% previous to the official announcement in the WSJ, indicating that it was more likely than not that he&#8217-d stay at the White House. So, in terms of absolute accuracy, that particular prediction market failed.

#2. Emile-Servan-Schreiber is right that, scientifically, we should assess a series of identical prediction markets, not just one, if we want to determine whether this market-based technology has merit. (And we should assess them comparatively to competitive institutions&#8217- predictions.) Overall, the NewsFutures prediction exchange is indeed predictive.

&#8212-

Just like the Olympic City prediction markets, the resignation prediction markets are rarely predictive because there aren&#8217-t any reliable advanced indicators to guide the traders. The Olympic committee is secretive and does not grant on merit but on politics (or corruption). As for the embroiled officials (politicians or CEOs), they are secretive too and send false signals (&#8221-Read my lips- I will never resign&#8221-). In both cases, the event derivative traders don&#8217-t have any access to inside information, the only one that counts. So these two types of prediction markets are of inferior quality, which explains why experienced traders don&#8217-t speculate on them. To have a better understanding of the prediction markets, in addition to the very good argument that EJSS makes, I think we should rate the advanced indicators. When they are of poor quality, we should disclose it to our readers and traders.

&#8212-

NEXT: Jed Christiansen strongly believes that Chris Masse has a bad understanding of probabilities.

Were the InTrade prediction markets on the November 2006s Senate elections accurate?

Posted on by
Reply

No Gravatar

Revisiting the issue, almost one year later.

Lance Fortnow (University of Chicago) wrote:

So how did those predictions go? In short you can say the markets predicted every individual race correctly but got the senate wrong, but let us look a little more carefully.

At about 9 AM CST on the morning of election day I made a snap shot of the map for a Discovery Channel Website article.

Every state colored blue was won by a democrat and every state colored red went to a republican. But also note the 69% given to GOP (Republican) Senate control although this election will give control to the democrats. No outcome would have made all the states and senate control agree with the 9 AM map.

Were the markets inconsistent? No, because the markets predict not absolutely but probabilistically. For example, the markets gave a probability of winning 60% for each of Virginia and Missouri and the democrats needed both to take the senate. If these races were independent events, the probability that the democrats take both is 36% or a 64% chance of GOP senate control assuming no other surprises.

Of course the races were not independent events and there are other states involved making it more difficult to compare the probabilities of the individual races with that of senate control.

So how did the markets do as predictors? Quite well as the outcome seems quite reasonable given the markets. Other outcomes would have also been reasonable such as the Democrats losing Virginia and the senate remaining in republican hands, a possibility that came very close to happening.

Previous blog posts by Chris F. Masse:

  • Professor Koleman Strumpf explains the prediction markets to the countryland people.
  • Professor Koleman Strumpf tells CNN that a prediction market, by essence, can’t predict an upset.
  • Time magazine interview the 2 BetFair-Tradefair co-founders, and not a single time do they pronounce the magic words, “prediction markets”.
  • One Deep Throat told me that this VC firm might have been connected with the Irish prediction exchange, at inception.
  • BetFair Rapid = BetFair’s standalone, local, PC-based, order-entry software for prediction markets
  • Michael Moore tells the Democratic people to go Barack Obama in Pennsylvania (a two-tier state), but the polls and the prediction markets tell us that that won’t do the trick.
  • CALLING ALL DEEP THROATS: What is it that Smarkets want to do in Malta, E.U.? And what will Smarkets market anyway?

THE FATE OF HARRY POTTER IN J.K. ROWLINGS 7TH BOOK, THE DEATHLY HALLOWS: prediction market vs. bookmaker

Posted on by
4

No GravatarSPOILER ALERT: MIDAS ORACLE IS REVEALING WHO DIES IN HARRY POTTER AND THE DEATHLY HALLOWS. IF YOU DON&#8217-T WANT TO KNOW, STOP READING. (Hit your back button.)

&#8212-

Harry Potter and the Deathly Hallows

Harry Potter and The Deathly Hallows

&#8212-

Pioneer Press:

Here&#8217-s the good news: Harry [Potter], Ron, Hermione, Ginny and Hagrid do not die. But others do. The good guys who meet their demise include Fred Weasley- Tonks (the witch who loves to change her hair color) and her husband, werewolf Remus Lupin- and Tonks&#8217- father, Ted. Bad guys who die are Professor Snape- Bellatrix, Voldemort&#8217-s favorite servant- Crabbe, one of Draco Malfoy&#8217-s mean friends- and Pettigrew, Voldemort&#8217-s servant known as Wormtail. One of the saddest scenes follows the death of Dobby, the house elf Harry [Potter] freed in a previous book. Harry [Potter]&#8217-s beloved owl, Hedwig, also dies.

There is also an epilogue, which takes place 19 years later, revealing that Harry [Potter] married Ginny Weasley and Ron married Hermione. They meet at platform 9? to send their children to Hogwarts.

OK, but I would like more info about the epilogue. Does J.K. Rowling mentions that Harry Potter dies as a happy man at an old age? I think it&#8217-s important because, since the statement of the Harry Potter event derivative is vague (&#8221-alive&#8221-), that could be a factor in the expiry of the NewsFutures contract. (As for me, no matter what is said in the epilogue about Harry Potter dying as an happy man at an old age, I believe that Harry Potter is still &#8220-alive&#8221- if J.K. Rowling will be able to write a sequel to her 7th book.)

&#8212-

TEMPORARY CONCLUSION (until we get strong confirmation of the outcome of The Deathly Hallows from two other sources):

1. The NewsFutures event derivative was predictive (as was an Internet poll), thanks to comparative literature analysis (analysis of the past writings of J.K. Rowling).

2. The William Hill people were bull-shitting when they said that Harry Potter died and when they opened betting on who was the killer. They took 50,000 British pounds from suckers. 100% pure profit for William Hill. Niall O&#8217-Connor, who swallowed the William Hill P.R. bullshit, should come on Midas Oracle, concede defeat, and analyze the root of his debacle.

&#8212-

PREDICTION MARKET VS. BOOKMAKER: THE PREDICTION MARKET WON.

&#8212-


© NewsFutures

&#8212-

Static chart:

Harry Potter NewsFutures

&#8212-

THE NIALL O&#8217-CONNOR FESTIVAL (1 + 2):

[1] What is the conclusion to be drawn here? In a market such as this, the outcome will already be known, by certain people. Are we too assume that these individuals choose to bet with traditional risk-averse bookmakers such as William Hill, forcing said bookmakers to close their books on the event? And that the uniformed, who know nothing, but cannot believe that Harry Potter will be killed off, choose to bet on NewsFutures?

[2] My own opinion is that the insiders have no reason to trade on NewsFutures (they are not interested in the notion of bragging rights). Moreover, it is extremely unlikely that they are even aware of the prediction exchange. They are interested however, in being able to take 500 sterling of William Hill, on a [fairly] anonymous basis. And this it would seem is what they have indeed done. The fact that the NewsFutures market has not fallen into line, gives rise to the notion that it represents nothing more than an amalgam of uninformed guessers, who are ignorant in the psychology of traditional betting markets. If Harry Potter is killed off- there will certainly be a lot of explaining to do……

Well, sounds like it&#8217-s Niall O&#8217-Connor who will have &#8220-a lot of explaining to do&#8221-. :) &#8230- And sounds like Niall O&#8217-Connor is &#8220-ignorant in the psychology&#8221- of high-volume play-money prediction markets. :)

&#8212-

And Michael Giberson, too, should have some explaining to do. :)

&#8212-

DISCLOSURE: At times, I was a participant in this Harry Potter prediction market, but decided to get out some time ago because of the incertitude regarding how the epilogue would be taken into account in the expiry process.

&#8212-

NEXT: NEWSFUTURES JUDGES THAT HARRY POTTER IS STILL ALIVE AT THE END OF J.K. ROWLING&#8217-S 7TH NOVEL, THE DEATHLY HALLOWS.

Previous blog posts by Chris F. Masse:

  • Red Herring’s list of the top 100 North-American high-tech startups includes Inkling Markets —but not NewsFutures, Consensus Point, or Xpree.
  • Professor Koleman Strumpf explains the prediction markets to the countryland people.
  • Professor Koleman Strumpf tells CNN that a prediction market, by essence, can’t predict an upset.
  • Time magazine interview the 2 BetFair-Tradefair co-founders, and not a single time do they pronounce the magic words, “prediction markets”.
  • One Deep Throat told me that this VC firm might have been connected with the Irish prediction exchange, at inception.
  • BetFair Rapid = BetFair’s standalone, local, PC-based, order-entry software for prediction markets
  • Michael Moore tells the Democratic people to go Barack Obama in Pennsylvania (a two-tier state), but the polls and the prediction markets tell us that that won’t do the trick.