Super Bowl Analysis Highlights — Keith Jacks Gambles Second Turn

A bunch of Marginal Revolution commenters dared criticizing Keith Jacks Gamble&#8217-s Super Bowl Analysis Highlights (+ addendum). The impudence! The audacity! They called down the thunder- they should get ready for the boom. [*]

Yes, the results of individual plays depend on lots of factors including play selection, and there is no way my analysis can separate all of these factors. After all, football is a team sport, and thus I suspect that cleanly identifying a player&#8217-s sole contribution play-by-play to his team&#8217-s chance of winning may be impossible. Certainly the performance of the offensive line has a ton to do with a team&#8217-s success. But even with standard statistics for measuring a player&#8217-s performance these issues remain. Quarterbacks get credit for completions, passing yards, and throwing touchdowns even though the coach&#8217-s play calling, the line&#8217-s blocking, the receiver&#8217-s route running, ect have everything to do with creating those numbers. Attributing actions to the primary actors on a play is a natural way to compute statistics. At least my net probability points statistic provides some weight to the game situation when measuring a player&#8217-s impact on a given play. Certainly, a 3 yard run on 3rd and 2 at the opponent&#8217-s 4 yard line means more than a 3 yard run on 3rd and 10 at a team&#8217-s own 20. This difference isn&#8217-t captured in rushing yards, but it plays a major role in my numbers.

It&#8217-s not ludicrous to think that the error rate of a betting market is low. In fact, Professor Paul Tetlock&#8217-s research [PDF] shows that it is low. He finds that the implied probabilities (prices) for sports contracts on are very close to the frequency that these contracts payoff. See Figure 1 on page 41 and notice that for his data the differences between prices and frequencies are actually quite small- in the range of prices from 40 to 80 (where the prices were for most of the Super Bowl) he finds deviations of around 1 (1% in probability). Furthermore, a look at Table 1 on page 34 shows that the standard errors for the estimated deviations in this price range are too large to rule out &#8220-no deviation&#8221- as an unlikely truth.

True, before kickoff the betting market estimated that Florida had about a 30% chance of beating Ohio State, and certainly before kickoff the market would have estimated that Florida had only an very small chance of winning by so many touchdowns. However, I&#8217-m sure that the market&#8217-s estimate of Florida winning by so much was not zero. Thus, this one example doesn&#8217-t say much of anything about the error rate of betting markets. Small probability events do happen. I once saw a lady win $5,000 on a quarter slot machine in Vegas, but that doesn&#8217-t make it ludicrous for me to think that my chance of doing the same was extremely small.

I think that my analysis does account for bruising running, clock control, and ball control. If a Rhodes run wears on the defense, the market sees this fact and will raise the probability of the Colts winning more so than if he had just fallen down at the same spot without knocking into a defender. Also with clock control, if a Colts receiver stays in bounds to keep the clock moving to protect a lead, then the market sees this fact and will raise the probability of the Colts winning more so than if he had run out of bounds at the same spot. Certainly, Addai not fumbling will raise the probability of the Colts winning more so than if he had not controlled the ball. The reason that you don&#8217-t see these factors greatly boosting Rhodes&#8217- and Addai&#8217-s numbers in my analysis is that these things are to be expected of any NFL running back. All running backs pound defenders, stay in bounds when necessary, and hold on to the ball when most important. Thus, market prices only change a little when these actions are done successfully. Doing good things that are to be expected do not count that much towards a player&#8217-s performance in my analysis. However, doing something bad when something good is to be expected, such as fumbling, really hurts a player&#8217-s performance statistic. For example, Addai&#8217-s fumble and Manning&#8217-s interception hurt their overall net performance statistic.

I was surprised by how low Rhodes&#8217- performance measured in my analysis, and I think it&#8217-s in part due to poor play calling that unfortunately gets counted against Rhodes in my analysis. For example, when Rhodes ran for 8 yards on a 3rd and 10 inside the red zone, the market dropped the Colts chance of winning by 4.5% even though I think Rhodes getting 8 yards was a great outcome for a RUN inside the 10. The priced dropped because that play pretty much ended the Colts chance at scoring a touchdown on the drive given that the field goal team was now trotting onto the field. I think this price drop is suggestive that the Colts made a mistake in their play calling. There&#8217-s also a case in which Rhodes ran for 5 yards to midfield on a 2nd and 13, but the probability of the Colts winning dropped by 1%. Even though 5 yards is an above average outcome for a run, it&#8217-s not a good outcome for a play on a 2nd and 13, because the probability of a punt goes up. Although, not related to play calling, all of Rhodes&#8217- yards in the last 5 minutes of the game amounted in just a .5% increase in the Colts probability of winning because at this stage of the game the market was fully convinced that the Bears chance at a comeback was was less than 1%.

It&#8217-s a common misconception that betting lines are set up to get equal money on both sides. Betting lines are set up to maximize the sportsbooks&#8217- profits. See Steven Levitt&#8217-s paper [PDF] or just take a look at betting trends. NFL betters overbet on favorites, so I wouldn&#8217-t be surprised to see underdogs beating the spread a little more often than favorites. However, I would be shocked if underdogs can truly be expected to beat the spread more than 52.4% of the time since it would mean that an underdog better with a big pocketbook could expect to make a lot of money from the sportsbooks. Also, my data is taken from, an online exchange that takes no positions unlike a sportsbook. It&#8217-s a marketplace of betters, and so far, the evidence is that the prices there are pretty good estimates of probabilities. [&#8230-]

My plea to Yahoo! research scientist David Pennock

My Dear Honorable Doctor David Pennock,

Please, give us more choices. We want to be able to choose between:

real-world outcomes and Yahoo! search outcomes


– play money (Yootles) and real money (in the U.K., thru a BetFair patnership).

WE WANT FREEDOM, DOCTOR PENNOCK. DON&#8217-T IMPOSE YOUR PREFERENCES ON US. Ever heard of the Statue of Liberty that the French gave to your people, man??

Addendum: I almost forgot. And we want a high level of interactivity between blogs and prediction markets. Plus, we want imaginary prediction markets. Hurry up- we&#8217-re waiting.

Read the last blog posts by Chris Masse:

Icons. Geniuses. Mavericks. – REDUX

Last time, I told you about TED 2007 and I noticed Nathan Myhrvold (former MicroSoft CTO –he founded MicroSoft Research) in the speaker list. That prompted me to visit his site, Intellectual Ventures, home of his “invention company”:

Intellectual Ventures is an invention company. We conceive and patent our own inventions in-house through a world-renowned staff of internal and external scientists and engineers. We also acquire and license patented inventions from other inventors around the world. Our network of invention sources includes: large and small businesses, governments, academia, and individual inventors. These inventions span a diverse range of technologies including: software, semiconductors, wireless, consumer electronics, networking, lasers, biotechnology, and medical devices. Our current focus is on developing our invention portfolio. Over time, we intend to market our portfolio on a broad and non-exclusive basis through a variety of channels including spin-out companies.

In the November issue of IP Investor (PDF), it is said that Intellectual Ventures has filed about 400 patents, and Nathan Myhrvold slams eBay for having filled only “10 patents or 11 patents” —big companies need patents “for the future”, he says. It would be interesting to know how many patents BetFair has filled —BetFair, like eBay, is a peer-to-peer exchange.

Of course, once you have transitioned from “invention” to “innovation”, the road does not end there: more than 75% of new products fail.

All this to come back to Robin Hanson’s invention, the concept of decision markets (a market-generated automatism to extract the wisdom of crowds and make it a better decision tool than managers or politicians), which he spams at each conference he is invited in. As I wrote two times, it’s a solution in search for a problem. I would see only two applications (last time I said “only one”, but I got a new idea):

– As I said, series of decisions that are NEVER taken by senior executives.

– In the coming two decades, imaginary universes (like Second Life) are going to meet with prediction markets. Once we are there, we can imagine a Second Life sub-universe filled with libertarians, playing with imaginary prediction markets. Then, yes, if the gamers agree, imaginary decision markets could rule —both at the macro and micro levels.

More Resources On Innovation:

The Business Innovation Insider blog – (highly recommended, especially to my marketing guru Guy Kawasaki)

Innovators share the lessons they’ve learned during 2006 – (a hodgepodge of ideas taken from various thinkers)