Karl Rove will resign from the White House.
Emile Servan-Schreiber:
Chris, how exactly do you define “predictive”? If your criterion is “last trading price above 50%”, that would betray a very limited understanding of the nature of both probability and binary markets. That’s a debate you and I have had ever since the first days of chrisfmasse.com a propos the 2004 US presidential election.
To your credit, I don’t think anyone has yet proposed a good way of assessing the “predictiveness” (predictivity?) of a single binary market after the fact. It is a very difficult question. Does anyone here have an answer?
#1. What I see on the NewsFutures chart above is that the probability of Karl Rove resigning went to about 20% previous to the official announcement in the WSJ, indicating that it was more likely than not that he’-d stay at the White House. So, in terms of absolute accuracy, that particular prediction market failed.
#2. Emile-Servan-Schreiber is right that, scientifically, we should assess a series of identical prediction markets, not just one, if we want to determine whether this market-based technology has merit. (And we should assess them comparatively to competitive institutions’- predictions.) Overall, the NewsFutures prediction exchange is indeed predictive.
—-
Just like the Olympic City prediction markets, the resignation prediction markets are rarely predictive because there aren’-t any reliable advanced indicators to guide the traders. The Olympic committee is secretive and does not grant on merit but on politics (or corruption). As for the embroiled officials (politicians or CEOs), they are secretive too and send false signals (”-Read my lips- I will never resign”-). In both cases, the event derivative traders don’-t have any access to inside information, the only one that counts. So these two types of prediction markets are of inferior quality, which explains why experienced traders don’-t speculate on them. To have a better understanding of the prediction markets, in addition to the very good argument that EJSS makes, I think we should rate the advanced indicators. When they are of poor quality, we should disclose it to our readers and traders.
—-
NEXT: Jed Christiansen strongly believes that Chris Masse has a bad understanding of probabilities.
Secrecy? False Signals? Lack of inside information?
Sounds like the Harry Potter markets to me. I guess that was why experienced traders stayed away.
I think ESS is absolutely right as you point out in claim #2, above.
But the claim in #1 is fundamentally flawed.
What do you mean by the term “absolute accuracy”? Is a normal six sided dice absolutely accurate only if you roll exactly the expected value of 3.5 each time? Or is it only good if you roll a number “close enough” to 3.5 (say 3 or 4), and therefore you take a roll of 1 or 6 as symptomatic of a problem to be explained.
Wait. In the Harry Potter case, the literature experts were able to analyze the past (the past writings from J.K. Rowlings) and the NewsFutures prediction market did output an objective probability (around 75% climbing later to 85%).
Is that OK if I call “absolute accuracy” the determination about whether a prediction market got it right or wrong, and “relative accuracy” whether a prediction market got it better or worse than a competitive institution?
Secrecy? False Signals? Lack of inside information?
Sounds like the Harry Potter markets to me. I guess that was why experienced traders stayed away.
I think ESS is absolutely right as you point out in claim #2, above.
But the claim in #1 is fundamentally flawed.
What do you mean by the term “absolute accuracy”? Is a normal six sided dice absolutely accurate only if you roll exactly the expected value of 3.5 each time? Or is it only good if you roll a number “close enough” to 3.5 (say 3 or 4), and therefore you take a roll of 1 or 6 as symptomatic of a problem to be explained.
Wait. In the Harry Potter case, the literature experts were able to analyze the past (the past writings from J.K. Rowlings) and the NewsFutures prediction market did output an objective probability (around 75% climbing later to 85%).
Is that OK if I call “absolute accuracy” the determination about whether a prediction market got it right or wrong, and “relative accuracy” whether a prediction market got it better or worse than a competitive institution?