Critics of the efficient market hypothesis are all hat and no cattle.

No Gravatar

Robin Hanson:

EMH (at least the interesting version) says prices are our best estimates, so to deny EMH is to assert that prices are predictably wrong. And for EHM violations to be relevant for regulatory policy, price errors must be so systematic as to allow a government agency to follow some bureaucratic process to identify when prices are too high, vs. too low, and act on that info.

So the clearest way for EMH skeptics to show they are right is to collect a track record showing that they can predict, ahead of time, when prices are too high, vs. too low. There’s little point in picking out some year old event, and saying, “see that price drop was too big.” Monday morning quarterbacking is way too easy.

But if just before a price drop you’d been on record saying the price was too high, or if just after you’d said the price was too low, well then we could include your purported error in a EMH-skeptic track record. And with enough skeptics identifying enough purported price errors, it wouldn’t take long to collect enough data to see if EMH skeptics really do have a system for identifying price errors. (Of course some would do well just by chance, so we’d need to look at the whole set.)

With a proven skeptic track record, we could then begin a conversation about whether their system was the sort that regulators should embody in some official government process, in order to improve our financial system. (Or whether skeptics should just post their errors, and let speculators fix prices.)

But all this continual harping year after year on how EMH is obviously wrong, based on selective stories of past prices you say were obviously wrong, sounds awful suspicious when you don’t bother to publicly flag price errors at the time, much less to collect and publicize a track record of such error flags. (E.g., care to declare which prices are wrong today?) What’s up with that?

The Robin Hanson manipulation papers make unrealistic assumptions, but its not like prediction markets are a bad idea…!!…

No Gravatar

In terms of unrealistic assumptions in Robin Hanson&#8217-s series of papers on manipulation, the major ones have been out there since at least 2004.

Despite some limited evidence, the insistence on traders needing to know the direction of manipulation isn&#8217-t too compelling since the direction will be manifest insofar as the price is &#8220-wrong.&#8221- &#8220-Noise trader&#8221- is a politically loaded and misleading term. Misleading because it suggests that the mean effect will be zero, when in reality &#8220-noise trading&#8221- usually takes the form of feedback trading. Lack of feedback trading is a significant assumption in the Hanson manipulation papers. Fortunately, prediction markets have objective settlements at specified times, unlike traditional assets where the meaning of prices is open to interpretation, making them more prone to feedback trading and irrational booms and busts.

With prediction markets, conditions for manipulation are more favorable when the settlement is far off in time, and when there are subjective inputs to the settlement, e.g. in politics. A distant settlement simultaneously makes it less clear what the real price should be, and delays manipulator losses because there is less incentive to correct price. At the limit, a manipulator could introduce a price distortion when a contract is launched, only to reverse position for small liquidity-related loss immediately before settlement, thereby destroying the markets &#8220-integral&#8221- of error over time.

Another big assumption, also identified by Paul Hewitt, is that traders have equal account sizes. But maybe this isn&#8217-t a huge problem if settlement is forthcoming, and maybe the issue could be mitigated with additional exchange disclosures, such as the standard deviation of position sizes in a given market. While this could discourage liquidity as large traders would become paranoid about their positions, it is essentially a &#8220-soft&#8221- position limit, and traders would be forewarned of one-sided markets (which could of course be the result of someone well-informed, but I &#8211- the google-anonymous* writer &#8211- would bet that more concentration comes with more error on average&#8230- this can be tested by someone with the data, of course maintaining trader anonymity)

Even accounting for long-term settlement, feedback trading, semi-subjective settlements, and account size imbalances, it seems one would have to abide to an overly rigid tenet of &#8220-do no harm&#8221- to hold that prediction markets are, on net, a bad idea. (Do no harm is of course abhorrent to libertarians, and even doctors don&#8217-t actually follow such a rule.) Moreover, some pathologies like political self-fulfilling prophecy will only happen if prediction markets have already demonstrated their value and have become more popular. But even if one believes in their long term success, single pathologies can damage one&#8217-s reputation permanently&#8230- if one plans to die at a reasonable age.

[*Given the political climate, many firms have issued directives to employees to not engage in even the slightest appearance of impropriety, which might include blogging on manipulation.]

Financial Fiasco: How Americas Infatuation with Homeownership and Easy Money Created the Economic Crisis

No Gravatar

&#8220-A loose monetary policy that created lots of cheap money, government interventions into the housing market, and the hubris of Wall Street firms deemed &#8216-too big to fail&#8217- combined to send the world economy into a tailspin, argues Swedish author Johan Norberg.&#8221-

&#8220-Financial Fiasco: How America&#8217-s Infatuation with Homeownership and Easy Money Created the Economic Crisis, an overview of what caused the current financial crisis (and what did not) and how politicians of all parties and all ideologies helped make the problem much worse.&#8221-

Share This:

If Michael Giberson is wrong, then that means that Chris Masse is right.

Paul Hewitt:

I donta€™ know that you could say Chicago was the a€?weakest linka€?, just because it got dropped first in the voting. The political process caused it to go early. However, Michael Giberson is wrong to imply that the prediction was accurate on the basis that Chicago and Rio were fairly close. Leta€™s keep in mind that the options are about as discrete as they come. Even if Chicago were to have come in a close second, it would have been a complete miss by the market.

If one needed to make a decision that depended on whether Chicago would win the bid, the prior choice would have been completely wrong, once the true outcome was revealed.

I have to agree with Chris. The market participants did not possess a sufficient level of information completeness to arrive at the correct prediction. Furthermore, the discrete nature of the outcomes made it a risky prediction. Finally, Ia€™m guessing that few, if any, of the IOC voting members were involved in the prediction markets, leading one to conclude that all (or almost all) of the market participants were a€?noisea€? traders.

Elsewhere, another commentator claimed that, because the prediction market started to show Chicagoa€™s share falling during the morning of the vote, this was evidence that prediction markets work. Hardly. It does show that prediction markets rarely provide accurate predictions sufficiently in advance of the outcome, in order for useful decisions to be made.

The prediction market industry really needs to investigate the determinants of success and which types of markets (issues) have the potential to provide consistently accurate predictions. Way too much time and effort is being spent arguing about meaningless markets, trivial questions, and false accuracy claims.

Previously: The Chicago candidacy, which was favored by the prediction markets (and gullible bettors like Ben Shannon), is the one that fared the worst.

Previously: Chicago wona€™t have the Olympics in 2016.

ADDENDUM:

IOC

– BetFair&#8217-s event derivative prices:

chicago-olympics-betfair

– InTrade&#8217-s event derivative prices:

chicago-olympics-intrade

– HubDub&#8217-s event derivative prices:

Who will recieve the winning bid to host the 2016 Olympics?

Who has the best analysis for Chicagos failed bid for the Olympics?

IOC

Prof Michael Giberson:

I think the a€?small, secretive committeea€? explanation is weak [].

Bradbury does an excellent job sifting through the shifting coalitions revealed in the three rounds of IOC voting. Neither Madrid nor Toyko showed any significant ability to attract votes as the rounds proceeded. It was going to be Rio or Chicago all along, but Chicago was weakest in the four-way vote and lost early, leaving the games to go to Brazil.

Based on Bradburya€™s [analysis], Ia€™m convinced that the decision was pretty much a toss up between Chicago and Rio. That conclusion was also implied in the prediction market prices just before the decision. Sure, the prediction markets favored Chicago, slightly, over Rio- I dona€™t think you can call it a miss given the closeness of the decision.

Well:

  1. The voting mechanism of the IOC regarding the 2016 Olympics venue was known to the news media and the prediction market traders (like Ben Shannon) well before the vote.
  2. The prediction market traders gave a surreal boost to the Chicago probability.
  3. The reality check is that Chicago was the weakest candidate.
  4. Hence, the prediction market traders were not informed enough about the basic facts regarding the IOC voting, for the reason that the International Olympic Committee is governed by secrecy, politics, and pork.

Next: &#8220-I have to agree with Chris. The market participants did not possess a sufficient level of information completeness to arrive at the correct prediction.&#8221-

Previously: The Chicago candidacy, which was favored by the prediction markets (and gullible bettors like Ben Shannon), is the one that fared the worst.

Previously: Chicago wona€™t have the Olympics in 2016.

ADDENDUM:

– BetFair&#8217-s event derivative prices:

chicago-olympics-betfair

– InTrade&#8217-s event derivative prices:

chicago-olympics-intrade

– HubDub&#8217-s event derivative prices:

Who will recieve the winning bid to host the 2016 Olympics?

Why an analyst should assess each newly created prediction market

IOC

The Chicago candidacy, which was favored by the prediction markets (and gullible bettors like Ben Shannon), is the one that fared the worst.

As we have blogged here many times, not every prediction market is created equal. Some are bound to aggregate bits of known information. Some others (e.g., the Olympic city prediction markets) are not able to do that, because no good information is leaking out. The IOC is a close aristocratic group that does not leak out good information. Those who forgot that and bet the farm on Chicago are now licking their wounds. You need an information analyst to assess whether a particular prediction market is pertinent.

– BetFair&#8217-s event derivative prices:

chicago-olympics-betfair

– InTrade&#8217-s event derivative prices:

chicago-olympics-intrade

– HubDub&#8217-s event derivative prices:

Who will recieve the winning bid to host the 2016 Olympics?

Chicago wont have the Olympics in 2016.

IOC

The Chicago candidacy, which was favored by the prediction markets (and bettors like Ben Shannon), is the one that fared the worst.

I TOLD YOU SO:

&#8220-Will Chicago get the Olympics? Dona€™t bet on it. Too risky.&#8220-

The prediction markets are not able to forecast which country will get the Olympics. The IOC is a close aristocratic group that does not leak information. Hence, it is not possible to aggregate information.

Once again, Ben Shannon made a very bad bet. He should read Midas Oracle more often &#8212-if he wants to avoid personal bankruptcy.

– Once again, we see that the P.R. agents of InTrade and BetFair (who both bragged about being able to predict Chicago) were overselling.

BetFair&#8217-s event derivative prices (on the far right of the chart, you can see that the price went down to zero):

chicago-olympics-betfair

InTrade&#8217-s event derivative prices (on the far right of the chart, you can see that the price went down to zero):

chicago-olympics-intrade

– HubDub&#8217-s event derivative prices:

Who will recieve the winning bid to host the 2016 Olympics?

Assessing the usefulness of enterprise prediction markets

No Gravatar

Do you need to have experience in running an enterprise prediction exchange in order to assess the pertinence of enterprise prediction markets?

Paul Hewitt:

Hi Jed…

As for qualifications, I have been making business decisions for almost 30 years. I am a chartered accountant and a business owner. Starting in university and continuing to this day, I have been researching information needs for corporate decision making. As Chris points out, I’m not a salesperson for any of the software developers. In fact, if I have a bias, it is to be slightly in favour of prediction markets. That said, I still haven’t seen any convincing evidence that they work as promised by ANY of the vendors.

As for whether I have ever run or administered a prediction market, the answer is no. Does that mean I am not qualified to critique the cases that have been published? Hardly. You don’t have to run a PM to know that it is flawed. Those that do, end up trying to justify minuscule “improvements” in the accuracy of predictions. They also fail to consider the consistency of the predictions. Without this, EPMs will never catch on. Sorry, but that is just plain common sense.

The pilot cases that have been reported are pretty poor examples of prediction market successes. In almost every case, the participants were (at least mostly) the same ones that were involved with internal forecasting. The HP markets, yes, the Holy Grail of all prediction markets, merely showed that prediction markets are good at aggregating the information already aggregated by the company forecasters! They showed that PMs are only slightly better than other traditional methods – and mainly because of the bias reduction. Being slightly better is not good enough in the corporate world.

I think I bring a healthy skepticism to the assessment of prediction markets. I truly want to believe, but I need to be convinced. I am no evangelist, and there is no place for that in scientific research. Rather than condemn me for not administering a PM, why not address the real issues that arise from my analyses?

Paul Hewitt&#8217-s blog

Previously: The truth about CrowdClarity’s extraordinary predictive power (which impresses Jed Christiansen so much)

The truth about CrowdClaritys extraordinary predictive power (which impresses Jed Christiansen so much)

No Gravatar

Paul Hewitt:

At first blush, it appears that we finally have a bona fide prediction market success! If we&#8217-re going to celebrate, I&#8217-d suggest Prosecco, not Champagne, however.

There are a number of reasons to be cautious. These represent only a couple of markets. We don&#8217-t know why Urban Science people appear to be so adept at forecasting GM sales in turbulent times. There is no information on the CrowdClarity web site to indicate why the markets were successful nor how their mechanism might have played a role in the PM accuracy. I&#8217-m guessing that it would have been really easy to beat GM&#8217-s forecasts in November, as they would likely have been even more biased than usual, mainly for political reasons. I&#8217-m not sure how Edmunds.com&#8217-s may have been biased or why their predictions were not accurate. Maybe they are not so good at predicting unless the market is fairly stable.

The CrowdClarity web site boasts that a few days after the markets were opened, the predictions were fairly close to the eventual outcome. This is a good thing, but, at this point it is not useful. No one knew, at that time, that those early predictions would turn out to be reasonably accurate. As a result, no one would have relied upon these early predictions to make decisions.

I&#8217-m even more skeptical of the company&#8217-s contention that markets can be operated with as few as 13 participants. Here we go again, trying to fake diversity.

It is interesting that a prediction market comprised of participants outside of the subject company did generate more accurate predictions than GM insiders (biased) and Edmunds.com (experts). The question that needs to be answered is why. Clearly, Urban Science people did have access to better information, but why?

Unless we know why the prediction markets were successful at CrowdClarity, it is hard to get excited. There are too many examples of prediction markets that are not significantly better than traditional forecasting methods. This one could be a fluke.

I&#8217-ll have more to say, soon, when I write about the prediction markets that were run at General Mills. There the authors of the study found that prediction markets were no better than the company internal forecasting process.

Paul Hewitt&#8217-s analysis is more interesting than Jed Christiansen&#8217-s naive take.

Paul Hewitt&#8217-s blog

Next: Assessing the usefulness of enterprise prediction markets

Share This:

Finally, a positive corporate prediction market case study… -well, according to Jed Christiansen

No Gravatar

Jed Christiansen:

To recap, the prediction market beat the official GM forecast (made at the beginning of the month) easily, which isn’t hugely surprising considering the myopic nature of internal forecasting. But the prediction market also beat the Edmunds.com forecast. This is particularly interesting, as Edmunds would have had the opportunity to review almost the entire month’s news and data before making their forecast at the end of the month. […]

Assume that even with three weeks’ early warning Chevrolet was only able to save 10% of that gap, it’s still $80million in savings. Even if a corporate prediction market for a giant company like GM cost $200,000 a year, that would still be a return on investment of 40,000 %. And again, that’s in the Chevrolet division alone. […]

Make up your own mind by reading the whole piece.

Next: The truth about CrowdClarity’s extraordinary predictive power (which impresses Jed Christiansen so much)

Next: Assessing the usefulness of enterprise prediction markets