The Accuracy Of Prediction Markets

A Lesson in Prediction Markets from the Game of Craps &#8211- by Paul Hewitt

Why Public Prediction Markets Fail &#8211- by Paul Hewitt

Both articles are required reading for Jed Christiansen and Panos Ipeirotis (alias &#8220-Prof Panos&#8221-). :-D

24 thoughts on “The Accuracy Of Prediction Markets

  1. Medemi said:

    I already mentioned “they” should be called probability markets in stead of prediction markets. There, that summarizes your first article.

    It is Chris? who calls them prediction markets and has the nerve to tells us they fail to predict. I don’t know what this guy wants.

    As for your 2nd article, I think it is reasonable to assume that the betfair markets you mentioned are pretty accurate, indeed, because the betfair markets are very efficient in general. Betfair even allow the insiders to bet.

    Accuracy of course, we’ve talked about this, can only be measured when we have recurring events/markets resulting in massive loads of data to interpret/analyze. I don’t think I have to waste much more of my precious time on this either.

    Good articles.

  2. Medemi said:

    On betfair, you won’t find a single soul pretending that the markets you bet on are somehow predictive. The word “prediction” is never used, most people probably never heard of “prediction markets”. People know it’s a gamble and they understand the odds.

    What most people don’t know, is that you haven’t got a chance in hell of beating the markets unless you have possession of superior information in one form or another.

  3. Medemi said:


    maybe you can do an article on betting integrity one day. The effect of insider trading for instance on market efficiency/accuracy. This is where the discussion really gets interesting.

    My opinion, in a nutshell, is this. What is reality? Reality is the collection of all pieces of information which are accessible, which are in the public domain. Information that is privy (inside information) should stay where it is and should not find it’s way to prediction markets or any other place. Well, reality is relative is my point I guess.

    I see trading on inside information as a threat to the future of prediction markets.

    1. It’s a threat to sports integrity (betfair’s case) and morality in general.

    2. I’ve seen many knowledgable traders with superior information (who help the markets become more efficient) get hurt by insider scum. I’ve been hit many times myself in my trading days, albeit in the stock market. Knowledgeable punters get hit disproportionally which poses a threat to prediction markets almost by definition.

    Insiders leave behind a path of destruction, to traders, betting exchanges and society as a whole, yet most people in the PM industry seem to welcome them. It’s a killer in the long run IMO, it destroys the dream of being able to beat the market and make a living using nothing other than sheer inventiveness. In the real world you will be rewarded massively for this, PM”s on the other hand will punish you. That’s not how it is supposed to be.

    The dilemma is this – most people don’t understand the destructive forces of insiders because there are not so many knowledgeable players getting hurt by them. Betfair can get away with it because most people are gamblers anyway, but what about the future of PM’s ?

  4. Niall O'Connor said:

    I think it is really time to end the bullshit and to go back to calling these entities what they are – (illiquid) Betting Markets.

    A betting market is a market in which people speculate on their beliefs. The market price that finally emerges constitutes an aggregate opinion, incorporating all of the information available to each of the participants.

    Comparing the empirical winning chance with the proportion of money bet on the horses, the market odds have been shown to be highly correlated with the empirical odds. The finishing positions of the horses fall monotonically in the direction predicted by the odds.

    In general, the betting market’s predictive ability is not something that punters knowingly concern themselves with. This is a situation that is unlikely to end anytime soon.

    To all intents and purposes, prediction markets are nothing more than an over-hyped joke.

  5. Medemi said:

    >> I think it is really time to end the bullshit and to go back to calling these entities what they are – (illiquid) Betting Markets.

    You have my vote, LOL. When I first visited Midas Oracle I thought “what the hell are these folks talking about?” :-D

    I see a bigger future with respect to EPM’s etc. though.

  6. Niall O'Connor said:


    You are great at talking the talk, but do you bother to read the literature?

    The presence and effect of insiders on betting markets is well documented, although one does not get that impression from your rants. You talk as though you alone are aware of the presence of insiders in said markets. If there were no insiders, from where else would the market obtain its predictive ability?

    May I point you towards the work of Shin, H.S, Crafts, N.F.R or perhaps you could beging by reading “Insider Trading, Herding Behaviour and Market Plungers in the British Horse-race Betting Market by: David Law, David A Peel.”

    Insiders are nothing new – they have always been there and will always be there. It is your job to devise a profitable betting strategy that exploits that fact.

  7. Medemi said:


    what the hell do theorists, scientists and the like know, eh?

    I used to know two succesful players (who provide massive liquidity on betfair) who had to give up close to 50% of their profits to insiders.

    Personally, I got an invitation to join a class action lawsuit 7 times during the course of one year. I don’t trade the stock market these days.

    >> It is your job to devise a profitable betting strategy that exploits that fact.

    That’s what I mean when I say it is a threat to morality in general.

  8. Medemi said:

    >> Prediction Markets = Betting Markets

    The usage of the term “prediction markets” does give the wrong impression to people who are not familiar with it. You know how stupid people are. It gets a lot worse for people who have no statistics background. My feeling is that none of us should really be surprised about the misinterpretation in the media and the hype and backfire that followed.

    I suppose we can’t change anything now.

  9. Paul Hewitt said:

    Hi Medemi…

    Personally, I wouldn’t call these markets prediction markets, either. The Betfair markets are really gambling markets that *may* be able to predict (like horse race markets). The Hubdub markets are more like polls, run for entertainment purposes. I’m not a big fan of either for prediction purposes. That’s why my focus is on enterprise prediction markets. I was only looking at these public markets to try to understand why they did not work.

    As for insiders, I think they are necessary for prediction markets, but a problem for gambling and real stock markets. Prediction markets work best when the participants (collectively) have access to the “complete” information set. Inside information is part of this. In gambling markets, the insiders will take advantage of those who don’t have the correct information. This makes it an unfair advantage, which threatens the market as a whole. That’s why we have insider trading laws.

    In the usual casino gambling “markets”, the odds are known to all. There is no “insider” knowledge. The outcome distributions are known with certainty. Consequently, they are “fair” games. Gambling on sports and other events may be subject to insider knowledge. One could argue that the final odds will, still, reflect all knowledge (i.e. be well-calibrated), but it will not be a “fair” game to all. Such “markets” would also have to be “efficient”. Unless I am the insider, I would avoid all such markets (with real money).

  10. Chris F. Masse said:


    Prediction Markets = Betting Markets = Event Derivative Markets

  11. Niall O'Connor said:


    I do not share your taxonomy.

  12. Chris F. Masse said:



    Prediction Markets = Betting Markets = Event Derivative Markets :-D

  13. Medemi said:


    on what basis can you make a clear distinction between gambling markets and Enterprise prediction markets?

  14. Paul Hewitt said:


    I’m not saying it is a very clear distinction between the two. However, Betfair markets are set up for the primary purpose of betting. Their predictive value is a side benefit (to the extent they work). My point was that some may work, but these discrete outcome markets are not very good for use in decision-making. Thankfully, there are few decisions that depend on who wins American Idol! EPMs *are* set up for the primary purpose of predicting. The gambling aspect arises as a result of the market design. With EPMs the focus is on the accuracy of the prediction. In an EPM winner-take-all market, even if the “frontrunner” fails to “win”, the second best share *may* be good enough (it will never be good enough where there are discrete outcomes).

  15. Medemi said:

    You could also argue that EPM’s only have value for a couple of thousand people on certain occasions while betfair markets could easily add value for 100 million people.

    Betfair is rightly criticized for not exploiting the social utility of prediction markets. A pity, because they have the right stuff – customer base, liquidity, trader diversity and proper incentive. It was my underlying motivation for criticizing them for so long. Likewise, I suppose we can criticize EPM’s for being so stubborn, asif one can demand accuracy simply because there is a wish for it. It’s not that simple. I witnessed a carpenter with a broken back transform into a professional gambler. When you see things like this happen you know you’re getting somewhere with respect to accuracy.

  16. Chris F. Masse said:

    Primary purpose… secondary purpose… All this is B.S. distinction.

  17. Chris F. Masse said:

    I stand by the 2 ideas I published there. But what was (in your mind) the relationship between that post and the present post?

  18. Medemi said:

    The Dutch and Swiss governments have decided there’s an urgent need to protect it’s citizens form gambling sites like betfair. Betfair is also the first company mentioned when you look up “prediction markets” on wikipedia.

  19. Panos Ipeirotis said:

    “these discrete outcome markets are not very good for use in decision-making”

    I would actually argue strongly against this. What if you have an event that is indeed binary? (Say, GM will file for bankruptcy.) Credit default swaps are fundamentally such contracts and they serve a useful purpose for hedging, if used properly. For example, prices of such contracts can be used to evaluate the riskiness of a portfolio, without waiting for the rating agencies to re-evaluate the trustworthiness of a country, city, company, etc.

    Niall, the name for the market is customized according to the intended audience. Target the “mass”? Call it a betting/gambling market. Target investors? Call it a futures market. Target conservative households? Call it insurance…

  20. Niall O'Connor said:

    Target the brainless …call it prediction markets…

  21. Chris F. Masse said:

    Good Irish humor… but a bit over the board.

  22. Panos Ipeirotis said:

    Niall, I see your point, in general. For example, outcomes of TV shows, or sports games, are indeed pure betting markets and should be called that.

    There are contracts though that can be useful (future price of oil, future interest rates, etc) that can serve a useful purpose, as explained above. Can they be used for gambling? Of course. The whole housing market mess and the role of the credit default swaps is widely documented, no need for me to add anything. But I would still be hesitant to call them gambling.

    Now, do we need to call them *prediction* markets? I would agree that there is no real reason for that… All markets have information aggregation characteristics and sometimes it is annoying to see how much we reinvent the wheel when studying “prediction” markets, pretending that they are fundamentally new beasts. (The “we” includes myself.)

    If we called them simply markets, I think we would have made significantly faster progress…

  23. Niall O'Connor said:


    Here, Here.

    I do not for one moment anticpate that Chris will relinguish his useage of the term

    “Prediction Markets.” Nonetheless, I feel that overtime, he may come to finally appreciate that the term actually serves to do more harm than good.

Leave a Reply

Your email address will not be published. Required fields are marked *