—

Based on that book by Carlota Perez, the blogger at The Park Paradigm says that BetFair is a 6th-paradigm company.

In her model, the world has experienced five technological revolutions and “techno-economic” paradigms in the modern era, starting with the Industrial Revolution in the late 18th century, through to the current end of 20th century paradigm – the Age of Information and Telecommunications. So what happens when we look past the present age to the next – the sixth – techno-economic paradigm? What are the new and emerging opportunities that will present themselves in this “sixth paradigm”? Have we already witnessed the innovation that will lie at the heart of the sixth paradigm or is it still ahead of us? Will the sixth paradigm be the “Age of Markets”?

That’s your job to list these specific situations, mister Hanson.

Psstt… I’m in London, U.K. They made me drink hot chocolate with floating marshmallows!!!… In France, it’s considered a crime. Anyway. I love London and the Londoners. Sky’s blue today.

Researchers use scoring rules as a laboratory tool to convince people to reveal their true expectations about some set of outcomes. Participants are asked to give estimates of the likelihood for a set of outcomes, their scores are some function of the value they gave for the actual outcome. Scoring Rules are called “Proper” if they are designed so the participant’s best strategy is to honestly reveal the probabilities that seem most likely. The Logarithmic Scoring Rule (one of the Proper rules) provides a reward that equals the logarithm of whichever estimate turns out to correspond to the actual value. Since the total of all the estimates must be 1, the participant can only increase some probabilities by decreasing others.

Robin Hanson described how an Automated Market Maker (AMM) that adjusts its prices based on a scoring rule can support unlimited liquidity in a prediction market. If each successive participant in the market pays the difference between the payoff for her probability estimate and that due to the previous participant, the AMM effectively only pays the final participant. If the AMM’s scoring rule is logarithmic, participants who only update some probabilities don’t effect the relative probabilities of others they haven’t modified. (This last effect is only valuable for Combinatorial Markets, which I’ll talk about in a later post.)

The change in the user’s payoff is log(newP) - log(oldP) (or equivalently log(newP/oldP)) for each state. For a binary question, the possible gain will be log(newP/oldP), and the cost will be log((1-oldP) / (1-newP)). For the rest of this article, I’ll use gain and cost rather than the log(...) expressions, since there are only these two, and I’ll be using them a lot.In multi-outcome markets, the most common approach is to let the user specify a single outcome to be increased or decreased, and to adjust all the other outcomes equally, but this isn’t the only possibility. This design choice has the useful property that the probabilities of other outcomes will be unchanged relative to one another. Since the other outcomes are treated uniformly, they can be lumped together, which results in the same arithmetic as a binary market. Since those other cases sum to 1-P, the price is cost. It is also reasonable to allow the user to specify either a complete set of probabilities, or particular cases to increase and decrease and how much to change them. Whatever the case, the LMSR adjusts the reward for each outcome to be log(newPi/oldPi). I’ll describe more possibilities in this vein when I cover the Combinatorial Market.

I hope you found all this interesting in an intellectual sort of way, but you may have noticed that this description isn’t applicable to markets in which the traders hold cash and securities. The whole thing is couched in terms of participants who will receive a variable payoff, but they don’t pay for the assets, they merely rearrange their predictions in order to improve their reward.

In order to turn this into an AMM that accepts cash for conditional securities, we have to pay careful attention to the effects of the MSR on people’s wealth. The effects are easiest to describe in the binary case, and every other case is directly analogous, so I’ll start there. In a binary market, the participant raises one probability estimate (call it A) from oldP to newP and lowers the probability of the opposite outcome (not A) from 1-oldP to 1-newP. If the trader had no prior investment in this market, the reward will increase by gain.

In order to reproduce that effect in cash and securities, the AMM charges cost in exchange for gain + loss in conditional securities. Why does the trader get securities equal to the cost plus the potential gain? The effect of this is that if A occurs, the participant has paid cost, and received gain + cost, for a net increase of gain over the original position. If A is judged false, the participant has paid cost with no return, which is the effect we hoped to match.

When an AMM supports a multi-outcome market using the approach I described above, one outcome is singled out to increase (or decrease), while all other outcomes move a uniform distance in the opposite direction. If the single outcome is increasing, the exchange is trivial to describe: we charge the trader cost for gain + cost in securities. The effect looks just like the binary case. The user has spent some money and owns a security that will pay off in a situation the trader thought was more likely than its price indicated.

If the trader singles out one outcome to sell (and thus reduce its probability), the difference among the alternatives I described in the first article in this series on Basic Prediction Markets Formats becomes evident. The trader is betting against something, and the market can represent this using short selling, complementary assets, or baskets of goods. The market might allow short selling (like InTrade), a complementary asset (like NewsFutures and Foresight Exchange), or a basket of securities representing all the other outcomes (like IEM). Since there are distinctly different points of view on this question, different markets will make different choices.

In order to support the short sales model, the trader needs to receive the payment first along with a conditional liability. In our model, the trader would receive gain in cash immediately, and securities that required repayment of gain + cost if the outcome (which the trader bet against) occurs. The platform would presumably require the trader to hold reserves to ensure the repayment.

With baskets of goods, the trader would get the appropriate number of shares of each of the other outcomes. The charge would be cost, and that would purchase gain + cost of conditional assets in all other outcomes.

The complementary assets model would charge cost in currency, and provide gain + cost of an asset that paid off if the identified outcome didn’t occur. The complicated part of this representation is that traders can hold both positive and negative assets. In a 4 outcome market, a trader holding 3 units of A and 2 units of B who sold 4 units of C could be shown equivalent portfolios of either A: 3, B: 2, C: -4 or A: 7, B: 6, D: 4. I think either choice is defensible. The first resembles the transactions the user has made, and so is probably more recognizable; the second provides a more consistent view of possible outcomes. (And looks the same as baskets.) If both positive and negative numbers are shown, the trader has to realize that the negative holdings pay off in all other cases. On the other hand, displaying a portfolio in a 7-outcome market as A: 3, B: 3, C: 3, E: 5, F: 3, G: 3 doesn’t seem as clear as D: -3, E: 2.

I doubt this detail will be of much interest to most users of Prediction Markets. Luckily for them, the trade-off the logarithmic rule makes between cost and reward just happens to produce prices that match probabilities. But if you are implementing Hanson’s LMSR, you should understand the alternatives well enough to verify that your market maker correctly implements the design. Zocalo Prediction Markets support binary and multi-outcome markets with a Market Maker based on the Logarithmic Market Scoring Rule. The design takes advantage of the parallels between the different markets by only implementing the logarithmic rule in one place.

This article is cross-posted from pancrit.org.

Other Articles in this series

• PM intro: basic formats (2005-12-30)
• PMs with Open-ended Prices (2006-01-05)
• Looking at Both Sides (2006-04-17)
• Book and Market Maker (2006-04-28)
• Liquidity in N-Way claims (2006-07-19)
• Continuous Outcomes using Bands and Ladders (2006-09-20)
• Integrating Book Orders and Market Makers (2007-01-10)
• Conditional and Combinatorial Betting (2007-03-06)

Different polling organizations have different strengths and weaknesses. Some use “tight screens” to scope out likely voters; others simply sample registered voters, without making any attempt to tighten the survey base to “likely voters.” Tight screening is especially crucial to gauge the true state of a primary, when committed base opinion can diverge significantly from less engaged moderate voters, and more importantly, influence those moderates over time to converge to the more partisan perspective. Some use human interviewers, although recently that has given way to IVR (Interactive Voice Recording) polls (the kind where a computer talks to you and asks you to “press 1 if you will definitely support X, 2 if probably…”)

I have found tight-screen, IVR polling to be the most reliable. IVR not only has no marginal cost, but it eliminates all the biases resulting from trying to give the most pleasant-sounding answer possible (the “sexy grad student effect” that exaggerated Kerry’s margin by 15 points in Pennsylvania 2004 exit polling, for example). IVR possible responses can also be randomly rotated from respondent to respondent to eliminate recency biases (first and last responses in a list exaggerated because those are at the forefront of a person’s memory of the list, not because s/he will vote that way).

The poster-child of IVR tight-screen polling success is Scott Rasmussen’s Rasmussen Reports. I have only tracked them over the last two election cycles (2004 and 2006), but considering that 2004 was a GOP wave and 2006 a Democratic wave election, I think the data is sufficient to form a valid judgment. Rasmussen’s track record is simply stupendous. It predicted 49 out of 50 states in 2004 correctly, usually within two percentage points of the actual outcome. In 2006, Rasmussen achieved similarly impressive results — all the more impressive when you consider that most polling models tend to err in favor of one party or the other. (“Likely voter” models tend to favor Republicans, and registered voter-based models tend to exaggerate Democratic strength.)

My other favorite sources include Gallup and Mason-Dixon. Gallup comes closer to the “registered voter” model than the tighter Rasmussen model, so Gallup usually lags tighter-screen polls. By election eve, however, the two models usually converge. Gallup’s election-eve congressional generic vote is hands-down the best in the business. However, their numbers for party primaries have poor predictive value, because they don’t make much effort to hunt down likely voters.

Differing survey methods can yield very different results. Rasmussen has long shown a much closer Democratic nomination race than most established, “registered voter” pollsters — most recently, it showed a 32-32 tie between Clinton and Obama, with Edwards wallowing 15 points behind. Gallup’s last numbers tightened drastically to a 31-26 race between Clinton and Obama (Gallup’s numbers are also hard to compare with Rasmussen’s because Gallup includes Gore).

Many smart Democrats, notably MyDD’s Chris Bowers, believe that Gallup and others are mistakenly including lots of “low information voters” who simply lag the opinions and thought processes of more-attuned Democratic partisans.

Now that more establishmentarian polling firms are coming in line with Rasmussen’s results, one can infer that the likely voter/ Chris Bowers theory has gotten the better of the argument.

A survey of pollsters wouldn’t be complete without knowing which ones to stay away from. Stay away from Zogby and CNN polling. James Carville’s and Stan Greenberg’s DemocracyCorps polling outfit is not trustworthy, either — for example, when they doubled the percentage of blacks in an October 2006 survey sample to bump the Democrats’ generic advantage by 5 points, to reinforce the Democratic narrative of a building wave.

Lastly, partisan pollsters in a competitive election season should always be taken with a grain of salt — they will use heuristic subtleties to create the best impression possible for their party’s candidates. Strategic Vision, a Republican outfit, deserves a three- or four-point handicap. Franklin Pierce generated a dubious Romney result for New Hampshire right after its lead pollster, Rich Killion, went to work for the Romney campaign. Such polls should be trusted only as a last resort.

For those of us who wish to divine movements in politics futures, discerning trustworthy data from bad data is paramount. Poll-rigging is the high art of Washington, DC, and as any interest group — or candidate — knows, it’s easier than easy to produce a poll that diverges wildly from reality, if the heuristics are threatening enough.

(cross-posted from my blog, The Tradesports Political Maven)