Midas Oracle .ORG » Mike Linksvayer

PredictionBook

Mike Linksvayer — Mon, 19 Oct 2009 07:33:37 +0000

PredictionBook is a new prediction tracking site, as described at LessWrong, which:

lets you make predictions and then track your calibration – see whether things you assigned a 70% probability happen 7 times out of 10.

The major challenge with a tool like this is (a) coming up with good short-term predictions to track (b) maintaining your will to keep on tracking yourself even if the results are discouraging, as they probably will be.

The site is a bit confusing–I have no idea what it means to “judge” others’ predictions of unresolved claims, but the idea of prediction tracking is an old one, touted with caveats for example by Robin Hanson (see here, here and here) and others, and it’s good to see a concrete attempt.

I’ve only looked at a few predictions, but one with many participants presents an interesting point of comparison with a roughly equivalent prediction market — “Barack Obama will serve a second term as president of the United States continuous with his present term.” I assume this prediction would count as true if Obama served one second of a 2nd term, rather than serving out a full 2nd term, but such are unmoderated claims.

The interesting part is that prior to my prediction, 24 were made ranging from 55% to 95%. My prediction?

mlinksva
estimated 53% and said “.647*.820 last trades for 2012.DEM.NOM.OBAMA and PRESIDENT.DEM.2012 on Intade.”
1 minute ago

Yep, that should be .820*.647 (the two claims respectively, though this doesn’t change the estimate) and I misspelled Intrade–ah well.

MBlume, the creator of the claim, estimated 80% and commented “I intentionally made this prediction prior to checking the intrade market.” One other user, Matt, said “I’ll track InTrade on this one.”, estimating 63%. However, Matt must have only checked PRESIDENT.DEM.2012 — four days ago when the estimate was made the last Intrade price was 63.1. 2012.DEM.NOM.OBAMA was 82.0. If he had wanted to track Intrade, he would have estimated .820*.631 = 52%.

For at least this one prediction on this one site, estimates are pretty crazy without the corrective of trading against crazy estimates.

I wonder if anyone has examined the accuracy of probabilities implied by multiple prediction market contracts such as the above? I would be fairly startled if such were inaccurate given the performance of the underlying prices that implied probabilities are derived from through multiplication, though perhaps biases seen with very low- or high-probability events would be exaggerated.

Midas Oracle = About + Archives + Authors + Best + Contact + Information Technology + Links + Probabilistic Predictions + Post-Mortem Predictions + Prediction Exchanges + Prediction Software + Publish Your Post + How To Publish

Intrade’s “Super Quick 8 Question Survey”

Mike Linksvayer — Sun, 06 Sep 2009 02:05:54 +0000

Intrade is promoting a “Super Quick 8 Question Survey” from their home page. [UPDATE: Results.] My answers:

Analysis of Barr and Nader 2008 Intrade contracts

Mike Linksvayer — Mon, 10 Nov 2008 00:02:37 +0000

I’ve used the Bob Barr contracts at Intrade to poke fun at the totally unrealistic expectations of Libertarian Party advocates (a couple times at Midas Oracle), so here’s a brief (and completely amateur) analysis of those contracts (and Nader contracts), post-election.

You may need to click on each chart to see the whole thing.

The probability of Barr obtaining 1% or more of the vote remained about .4 (40%) throughout the past several months. More optimistic scenarios became more discounted as the election grew nearer, and presumably it became clear Barr would not break through. Even 1% would have been seen as a breakthrough by LP advocates, but in the end Barr obtained only 0.4% of the vote. (Note that obtaining .4% of the vote and a .4 probability of obtaining 1% of the vote are very different things!)

Ralph Nader contracts attracted very little trading, though the 1% or greater contracts gave Nader a 60% chance of obtaining 1% of the vote as late as early October. Nader contracts for 3% and above did not trade at all — or almost not at all — Intrade’s web page table (screenshot below chart) shows a few trades, but no advanced chart or closing price/volume download, and there seemed to be an (unrelated?) possible bug with Nader contract reporting fairly consistently — last trade prices would not be remembered and reported in the aforementioned table — or it could be user (me) error/misunderstanding.

Both Barr and Nader contracts were traded heavily (for them) post-election, presumably as traders freed up cash and unwound positions — for unkown reasons Intrade still has not expired the contracts.

Spurred by comments from David Nolan (scan the page for “Intrade” or my name), I also attempted to gauge what traders thought about the average vote percentage candidates would receive across all scenarios — even a small chance of a genuine breakthrough could make an otherwise hopeless campaign (in the LP’s case, 9 such presidential campaigns prior to 2008) worthwhile. See below for the average (not most likely!) vote percentage over time each candidate might be expected to receive if the campaign were re-run may times. Assumptions: a floor of .5% (cases in which 1% is not met), very generous given that Barr did not reach even that, and only one LP candidate ever has, if candidate crosses threshold, they do so by .5%, also generous, and if 7% (the highest contract) is crossed, they obtain 7.5% of the vote, slightly ungenerous given the non-impossibility of obtaining a much higher percentage of the vote. Longshot bias should also expected to be at play. I don’t think these numbers should give LP or other third party advocates any comfort, though I admit my own bias on the matter. The average of all Barr scenarios declined steadily as the election approached, while Nader contracts did not trade until closer to the election, and they both ended at an average of 1% across all scenarios just before the election.

In the end Nader received 0.54% of the vote, beating Barr for (a very distant) third place behind Obama and McCain.

All of the data used above may be found in intrade-2008-barr-nader.zip. The spreadsheet file intrade-2008-barr-nader.ods aggregates everything.

2012

I’d like to see:

Fewer contracts (in 2008 there were 7 for Barr and Nader each, from 1% through 7%; actually 8 each counting an electoral vote contract that received almost no attention) might be better, and one testing 0.5% of the vote, which is a more relevant barrier for the LP (only broken once, in 1980) — maybe 0.5%, 1%, and 3%, or perhaps a single log-scaled contract.
Minor vote total contracts contingent on nominee — assuming they had attracted trades, would they have expected the more mainstream for US politics (but less mainstream among libertarians) Bob Barr to outperform other candidates for the LP nomination?
Even better than contingent nominee/vote total contracts, contingent nominee/some measure of welfare or liberty contracts. Presumably these would show no difference among LP nominees, on the theory that no such nominee will make a difference in the world.
More trades! Why are libertarians afraid of the market?

Futarchy Lite 2008

Mike Linksvayer — Mon, 03 Nov 2008 15:08:31 +0000

Below is a copy of a post yesterday on my personal blog. First, a couple asides for Midas Oracle readers.

After 2008

Two of Peter McCluskey’s four sets of contracts produced interesting results while all of Polimetrics’ were duds. (Not complaining; I’ll take the successes.) What should we take from this, other than a subsidy being helpful? (How helpful?)

It would be nice to see improved versions of the two successful sets offered for 2012 and perhaps for control of the US Congress in 2010. The Iraq contracts could be generalized to expected number of troops killed, number of troops abroad, or even size of the military budget.

It would be great to see conditional contracts for non-US jurisdictions as well.

On Intrade’s part, I’d really, really like to see implied outcomes and graphs of same built into the Intrade interface. Their lack is a huge barrier to understanding conditional contracts, both for journalists and bloggers who might come to appreciate their importance, and even for traders who need to understand them, even really smart traders.

Futarchy Lite?

Futarchy imagines that constituents or their representatives specify (presumably democratically) desired outcomes and prediction markets evaluate whether specific policies further desired outcomes, neatly captured in the phrase vote on values, bet on beliefs. One could imagine a variety of implementations, in particular concerning how policies are proposed, but that’s the basic framework.

So what does the guide have to do with futarchy?

It encourages voters to govern their own vote as a policy making body would govern policies. Presumably individuals have values or can decide on them, so there’s no voting, but one can use prediction markets to determine how they should vote in order to further their values. So the catchphrase for the futarchist voter guide below or what I’ll call “Futarchy Lite” is vote with your values, consensus beliefs. Ok, that’s not at all catchy. And I’m afraid to too many people interpret values and beliefs to be roughly the same, but that’s a bigger problem.

Mike’s Futarchist Voter Guide (also posted on Mike’s blog)

Four years ago I used play money contracts traded at the Foresight Exchange to provide a Futarchist Voter Guide (though I didn’t call it that). This U.S. election cycle relevant real money contracts are traded on Intrade.

The first set was instigated and subsidized by Peter McCluskey. Two have attracted a fair amount of interest and seem to be informative. They have consistently indicated that a Democrat will result in a smaller (but still approaching US$1 trillion!) increase in the US federal government debt over one year and a smaller number of US troops in Iraq. (The others, regarding the movement of oil and interest rate futures on election day, have shown no difference between expected election outcomes.)

Above: Expected increase in US Government debt between 30 Sep 2010 and 30 Sep 2011 if party wins US presidency.

Above: Number of US troops in Iraq on 30 June 2010 if party wins US presidency.

Note that briefly in early September the contracts indicate lower debt and fewer troops in Iraq with a Republican candidate. I suspect this is due to McCain’s brief surge following the GOP convention — the implied outcomes above depend on election winner contracts, and with a much lower volume, presumably take awhile to fully respond to rapid shifts in election outcome expectations.

A second set of relevant contracts instigated by Polimetrics have unfortunately attracted almost no trading and probably tell us nothing. Note however they also reflect the brief McCain surge, at which point they implied a greater than 100% chance of growth, low unemployment, and lower crime with a McCain win. They have since reverted to showing essentially no difference between Obama and McCain. Note that each series only starts when there have been trades.

Above: Percent chance that economic growth averages 2.5% or more for 2009-2011 if individual wins US presidency.

Above: Percent chance the US unemployment rate is less than 5.0% at the end of 2011 if individual wins US presidency.

Above: Percent chance the number of violent crimes committed in 2010 is lower than the number of violent crimes committed in 2007 if individual wins US presidency.

Peter McCluskey has automatically updated pages showing implied outcomes for each set of contracts given their latest trades.

(I intended to make a page with frequently updating graphs, but got lazy when Peter published the aforementioned pages, and only collected the data until now, which is available in a spreadsheet.)

Intrade 2008.PRES.McCAIN > PRESIDENT.REP2008

Mike Linksvayer — Thu, 11 Sep 2008 17:33:08 +0000

How frequent are arbitrage opportunities such as the following?

In addition to title, the reverse is true of OBAMA/DEM.

Do traders really think there’s some probability of McCain being elected as an idependent and Obama being replaced as the Democrat nominee?

Independent Institute’s weak blog post on prediction markets

Mike Linksvayer — Mon, 21 Jul 2008 01:34:32 +0000

I was happy to notice that the Independent Institute, the pro-market think tank that published Entrepreneurial Economics in 2002, featuring a Robin Hanson chapter on decision markets (and much else good), published a blog post titled Forget Polls: Look at Prediction Markets on the Election. Unfortunately, while the post mentions prediction markets, they are only used as a jumping off point for an oft-repeated and boring argument that voters ought consider candidates outside the dominant parties.

Sadly, I think the think tank question I posed at the end of 2006 is still no: When has a pro-market think tank ever subjected its policy recommendations to market evaluation?

So, I’ll extend the donation offer made in that post through the end of 2009.

Voodoo analysis of prediction market contracts

Mike Linksvayer — Sat, 28 Jun 2008 21:43:42 +0000

I wonder if the following is a joke:

Events these past few weeks make an airstrike on Iran more likely. The Intrade contract reinforces this view. While the probability remains moderate at 32%, the chart shows a market that is strengthening.

Here is stock-type technical analysis applied to this contract. There is a large “cup” going back to the contract’s inception. The low was 10 in January of this year. Since then, there is an unmistakeable rise. Following standard technical analysis, the drop from 50 to 10 was 40 points. That gives a calculated resistance level at 30. That level was broken this past week on high volume. This confirms the strength. The 40 level presents the next resistance level.

The last post mentioning technical analysis at Midas Oracle contains a joke.

Bob Barr markets

Mike Linksvayer — Thu, 26 Jun 2008 18:01:58 +0000

Reason, a libertarian periodical, writes that the Bob Barr effect is “confirmed.” Because Obama’s campaign manager says it is.

Yes, pathetically a pro-market publication heeds the remarks of a political operative rather than markets that say Bob Barr will not make an impact.

Admittedly we have very little signal from prediction markets and lots of noise from political operatives, so writing about the latter makes for easier journalism.

There are now Intrade contracts on Barr’s share of the popular vote. Perhaps they’ll provide a little more signal, but I don’t have high hopes for reasonable trading volume — or for libertarian politicos embracing markets when the message of market prices might not correspond to their hallucinations.

Bob Barr candidacy fails market test.

Mike Linksvayer — Mon, 26 May 2008 23:28:02 +0000

Yesterday at about 5:30PM EDT the Libertarian Party (U.S.) nominated ex-Congressperson Bob Barr for U.S. President. Barr’s nomination does not appear to have been certain — it took five rounds of voting, including two rounds where he tied for first and one in which in placed second.

So what do the relevant prediction markets make of this new information? Is Barr a contender, a potential spoiler, or irrelevant?

At Intrade, PRES.FIELD2008 has attracted no trades since May 22, three days before Barr’s nomination. We didn’t need a market to tell us a Libertarian Party nominee would not be a contender, nor help the chances of another non-Democrat and non-Republican.

The idea that Barr could be a spoiler is not completely ridiculous on its face (Barr and Wayne Allen Root, his running mate, are both recent ex-Republicans). However, PRES.DEM2008 has attracted no trades since May 24, the day before Barr’s nomination, while PRES.REP2008 did not trade between 18 hours before the nomination and over 3 hours after.

I think we can conclude that traders believe Barr’s nomination will have no impact on the outcome of the election. And, sadly, that volume on Intrade is pathetic.

Small comforts of prediction markets

Mike Linksvayer — Mon, 19 Nov 2007 06:58:08 +0000

Yesterday I had dinner with a friend I hadn’t seen for a few years. I asked what he’d been doing, apart from being a nerd, and he said he’d been spending too much time following the U.S. presidential campaigns (actually just the Ron Paul campaign, but that’s not particularly relevant here). I realized that I don’t do this anymore. It could be because I’m maturing, but I’ll give credit to prediction markets.

Most of the yapping in the media is about the horse race and personalities, which I don’t care about, other than the status of the former. Instead I check prices at Intrade most days, which gives me a more accurate and much more concise status update than any amount of time spent reading or watching commentary.

Furthermore, betting that candidates I detest will win and against candidates I mind less, even in small amounts, really helps me not waste time thinking (mostly distressed thoughts) about the election.

So thank you prediction markets for the time and peace of mind!