I agree that the simultaneity problem and the anticipation problem are confounding factors in assessing a player’s contribution to his team’s chance of winning. I’d like to say more on these issues.

(1) The Simultaneity Problem. Certainly, football is a team sport, and thus I suspect that cleanly identifying a player’s sole contribution play-by-play to his team’s chance of winning may be impossible. Certainly the performance of the offensive line has a ton to do with a team’s success. But even with standard statistics for measuring a player’s performance these issues remain. Quarterbacks get credit for completions, passing yards, and throwing touchdowns even though the coach’s play calling, the line’s blocking, the receiver’s route running, ect have everything to do with creating those numbers. Attributing actions to the primary actors on a play is a natural way to compute statistics when faced with the simultaneity problem. At least my net probability points statistic provides some weight to the game situation when measuring a player’s impact on a given play. Certainly, a 3 yard run on 3rd and 2 at the opponent’s 4 yard line means more than a 3 yard run on 3rd and 10 at a team’s own 20. This difference isn’t captured in rushing yards, but it plays a major role in my numbers. The Protrade win probability measure which similarly takes account of the game situation, also has this simultaneity problem.

Also on this issue of the simultaneity problem, there is at least one case in which a player’s contribution to his team’s chance of winning can be measured more cleanly, unexpected injuries and suspensions. For example, when Kevin Garnett was suspended for one game after throwing a punch (click here for the story), the drop in the odds of his team winning provides a nice measure of his impact.

(2) The Anticipation Problem. Certainly, the market builds in estimates of a team’s and players’ abilities. After all, the market gave the Colts a 68% chance of winning before they even stepped on the field. Certainly, this number says something about the Colts performance that isn’t captured in my net probability points statistic for each player. If the market expects that the Payton-Manning-led Colts offense will move the ball easily, then this expectation will be built into prices. As a result, Peyton Manning’s performance will indeed be understated by my measure relative to the performance of, for example, backup Jim Sorgi had he done the exact same thing as Peyton. However, it’s not so clear that Peyton’s performance will be understated relative to the performance of the Colts other offensive players. The anticipation problem holds for them as well. If the Colts offense is expected to move the ball easily, then receiver Reggie Wayne’s performance will also be understated relative to his performance without these high expectations.

Interestingly, if the market anticipated that the Bears wouldn’t have much success in moving the ball on the Colts, then this anticipation problem actually understates how bad Rex Grossman’s performance actually was! Despite the market’s low expectations of how the Bears would fair against the Colts, Rex contributed 36.5% (percentage points) to the Colts chance of winning by my measure, more than twice as much as any Colt. Just imagine how bad this number would have looked had the Bears actually been expected to beat the Colts.

There’s also a simultaneity problem with interpreting the market’s anticipation of the future as a pure problem. The fact that the market price incorporates expectations of the future can actually be an advantage for my measure. Just as the market price incorporates expectations of Peyton’s offensive ability, it also incorporates expectations of how bad the defense he’s playing against is. If the market expects the Colts offense to have success in part because the Bears’ pass defense is so poor, then my measure more accurately measures his performance than a measure that doesn’t take into account expectations of the Bears’ defensive ability. Certainly, a measure of a player’s performance should incorporate expectations of the defense’s ability. The Protrade’s current publicly available win probability measure doesn’t take expectations about particular teams’ defensive abilities into account, thus it will overstate the performance of offensive players when playing against a particularly poor defense.

I am big fan of Protrade’s attempts to measure a player’s performance in terms of his impact on the probability of his team’s chance of winning. However, estimates that only take into account historical data, and not market expectations, are bound to be inaccurate. Making estimates only taking into account historical data is like driving while looking in the rear-view mirror. There’s no problem as long as the road ahead is just like the road gone by. However, if the future isn’t like the past, then wham. (analogy courtesy Robert Merton). For example, just think about how the various new offensive strategies (recently, the spread in college football) can change the game. The market has an amazing ability to take into account all the particulars of the situation in a way that historical data cannot. Thus, I think probability estimates must take into account market expectations to measure accurately the probability impact of a play, which is essential to both of our measures of a player’s performance. The research director at Protrade assures me that they are working internally on a measure that takes team ability into account. I hope that their methods will be made public so that interested folks can see and test for themselves the accuracy of their measure.

In closing, I’m happy to see these probability based statistics getting more attention because I think they provide an exciting way to look at the game. Tom Brady provides an excellent example of the potential of these statistics. It’s a common notion that Tom Brady is better than his traditional statistics indicate because he has produced for his team in the moments when it mattered most. I’d love to see what a market-based probability performance measure would have looked like for Tom Brady in his many magical playoff performances and in his most recent dud.

SENTENCE OF THE DAY — Keith Jacks Gamble Edition — (Analogy from Robert Merton [*]):

Making estimates only taking into account historical data [and omitting market expectations] is like driving while looking in the rear-view mirror.

[*] Who the hell is Robert Merton????

Full Analysis

ABSTRACT: I analyze Superbowl XLI by matching price changes for a futures contract on the winner to play-by-play data. The price change following a play measures the impact of that play on the market’s expectation of which team will win. Thus, these price changes identify the relative importance of each play in determining the outcome. Four of the top five impact plays of the game were turnovers, led by Kelvin Hayden’s interception return for a touchdown. These price changes also provide a new statistic for measuring and comparing players’ performances. Despite winning the Most Valuable Player Award, Payton Manning ranks 10th on the list of positive impact players for the Colts. By far the greatest contributor to the Colts’ victory was Rex Grossman, whose poor play for the Bears contributed twice as much as the top performing Colt.

CONCLUSION: This analysis of Superbowl XLI shows how price changes on an online exchange can be used to measure the impact of each play on the final outcome. [...]

NOTE: This is just for the openers. It could be that Keith Jacks Gamble will publish a blog post on Midas Oracle, soon —if he wishes.

Over the last 22 Super Bowls, 18 — 86% of them — were won by the team that came into the game after allowing fewer regular-season points. (In the 2004-05 regular season, the New England Patriots and the Philadelphia Eagles each allowed 260 points.) Not since the Denver Broncos won 1999′s Super Bowl XXXIII has the team with the weaker defense emerged victorious.

And that’s why the underdog Chicago Bears should defeat the Indianapolis Colts in Miami on Sunday. Lovie Smith’s NFC Champion Bears allowed only 255 points during the regular season. That’s third-best in the NFL — and is a whopping 105 points fewer than the AFC Champion Colts allowed.

More bad news for Colts fans. In the regular season, the Colts defense allowed 360 points — more than any team that has won the Super Bowl. Since the advent of the 16-game schedule in 1978, only three winning teams allowed 300 or more points in a season — the 1983-84 Raiders (338), the 1998-99 Broncos (309) and the 1980-81 Raiders (306).

And while much has been made of the Colts’ powerful offense, the Bears scored the same number of points — 427 — in the regular season. Indianapolis quarterback Peyton Manning has been the focus of attention, but his Chicago counterpart Rex Grossman has been about as efficient in his passing this post-season — with 6.66 yards per attempt vs. Mr. Manning’s 6.84 yards per attempt.

That brings into view another trend pointing to a Bears win. A good gauge of a team’s strength is its ratio of points scored to points allowed — and the measurement we use to express that ratio is Pythagorean Wins. Since Super Bowl XIX, the team with more P-wins has gone 18-4 (81.8%) making it a more reliable indicator than a team’s actual record, which predicted the outcome just 76.5% of the time. This season, the 13-3 Bears outscored their opponents by 172 points (better than five of the last six Super Bowl champs) and led the NFC with 12.36 P-wins. The 12-4 Colts, on the other hand, outscored their rivals by just 67 points, which projects to only 9.59 P-Wins. No team in history has won the Super Bowl with a P-win total that low.

Instead of listening to the pundits who say their team has little chance against the Colts, maybe Chicago fans should focus on this: Only three teams since the 1970 NFL-AFL merger — the Raiders in Super Bowls XI, XV and XVIII — have entered the big game with fewer P-wins and a more porous regular-season defense than their opponents and still managed to win.

Savvy fans may dismiss many of these numbers, arguing that the AFC was the far better conference this year. That may be true for the conferences, but it doesn’t necessarily hold for this matchup. The Bears and Colts played five common opponents this season — the New York Jets, New York Giants, Patriots, Buffalo Bills and Miami Dolphins. Indianapolis went 5-0 against these teams, while the Bears were 3-2. However, in those games, the Bears scored 114 points and surrendered only 75, outscoring their opponents by 39 points. The Colts? Despite their perfect record, they scored 128 but gave up 107, for a 21-point differential, just over half that of the Bears.

I got long a little NFL.COLTS after they beat the Patriots. More recently, my boss asked me to get long a ton of NFL.BEARS, which is paying 2:1. Either way, I am happy.

Cross-posted from CaveatBettor.