N.F.L. Week 7 Game Probabilities: The Illusion of Power Rankings

Every major sports outlet seems to offer N.F.L. power rankings. I admit my site offers them, too. The game probability model I use for these weekly predictions also ranks teams. I match each team up against a notional league-average opponent at a neutral site, and rank each team according to the resulting game probabilities.

There’s a lot of randomness in football game outcomes, and the model I favor is designed to ignore the noise and focus on the signal. This approach can sometimes produce curious results, as teams with relatively poor records are often ranked well ahead of teams with better records. You might attribute that to the inaccuracies and limitations of statistics used, and that’s partly true. But there’s much more to the discrepancy.

Consider a world in which all N.F.L. teams were perfectly evenly matched. Every game would be little different than the flip of a coin. In this kind of world, there’s still a very good chance there would be at least one 5-1 team and one 1-5 team, despite both being completely equal in team strength. In this hypothetical world, a power ranking based on team record would be an exercise in self-delusion.

The real N.F.L. isn’t as far off from the National Coin Flip League as you might think. Certainly there are better and worse teams, but there is also randomness in game outcomes. So although team records are not complete illusions, they are partially random and often misleading, especially early in the season.

Most power rankings of one kind or another are heavily influenced by team record, and it’s easy to understand why. Even the sharpest and most diligent sports pundits, who have the luxury of watching a wall of simultaneous games on Sunday, can only take in a portion of the action. And they remember even less — their brains aren’t Tivos. The previous weeks’ action is an even fainter memory. And what they do remember is probably biased by the information they have at their fingertips, which is game scores. So it’s not surprising that most power rankings closely match team records.

The illusion of power rankings goes deep. Even if power rankings are not necessarily good predictors of future performance, they can still be sound predictors of final team records and playoff qualification. The reason is the wins-in-hand effect. Season-ending records and playoff qualification is a function of two factors: 1) to-date wins in hand, and 2) future win expectancy. Power rankings that are overly influenced by to-date wins in hand will appear to be far more accurate by the end of the season than they really are. Even objective quantitative rankings that over-fit to the noise of past performance will intuitively seem to be a much better gauge of team strength that they truly are. If you’re grading someone’s rankings according to how well they match final records, you’re letting them cheat. They’ve glimpsed half the answer key from the teacher’s desk.

We already know factor #1. The team standings are there for anyone to see. The tough part is figuring out factor #2, and that’s what the game probabilities below are based on.

Here are your game probabilities for Week 7:

Brian Burke, a former Navy pilot who has taken up the less dangerous hobby of N.F.L. statistical analysis, operates Advanced NFL Stats, a blog about football, math and human behavior.

N.F.L. Game Probabilities Are Back, With One Adjustment

Game predictions are back for 2011. As always, the probability model is based on passing, running, turnovers and penalty efficiency. But now, running is represented by Success Rate (SR) rather than Yards Per Carry (YPC). SR is the percentage of runs in which the a team’s point expectancy (based on down, distance and yard line) has improved.

SR correlates far better than YPC with winning games. YPC is too susceptible to a handful of relatively rare breakaway runs. I think of running as a jab and passing as a cross or uppercut. The jab is a low-risk punch that doesn’t expose your defenses, keeps your opponent off balance and guessing, and keeps him from purely defending against your cross. A good jab is a prerequisite, but the cross is what scores points and wins bouts.

Running can be more than that, of course. It’s essential in short yardage and inside the red zone. And when team has a lead, it burns clock and helps keep the ball out of an opponent’s hands in the fourth quarter. I believe the revised model captures this aspect of the running game and better reflects the true inner workings of the sport.

For new readers, here is refresher on how the model works. A logistic regression is fed net Yards Per Attempt (YPA), run SR, and interception rates on both offense and defense, plus offensive fumble rate. Team penalty rates (penalty yards per play) and home field advantage are also included. These particular statistics are selected because they are predictive of future outcomes, not because they explain past wins. This is a distinction overlooked by nearly all experts. Sometimes less really is more when it comes to predictions.

For example, turnover rates explain past outcomes very well, but a relatively small part of turnover rates carry forward and predict future outcomes. If a team has a very low interception rate of 1.2 percent, how likely are they to continue the season with so few interceptions? Chances are they will remain better than average, but not nearly as low as 1.2 percent. This concept is known as regression to the mean, and it’s essential for good predictions.

“Logistic regression” might sound like just mathy mumbo-jumbo, but don’t let it scare you off. The regression uses data from recent N.F.L. seasons to tell us how each facet of team performance is best weighted to predict which team will win a game. Each team variable is regressed again to account for how reliable each particular facet is throughout a season. In other words, the stats vary in terms of how consistent they are from game to game. For example, offensive passing efficiency is most consistent, and turnover rates are least consistent.

Lastly, the model adjusts for each team’s previous opponent strength. This is an especially important consideration early in the season, when some teams have only played weaker teams, and some have had to struggle against solid opponents. As the season wears on, strength of schedule will tend to even out, but never completely.

The predictions sometimes challenge our preconceived intuitions about games. If you knew nothing else about N.F.L. teams except for how well they’ve played so far this season, this is how you would want to handicap the games. Of course, we do know more than what we’ve seen the past three weeks. Sometimes that’s helpful, but at least as often it clouds our minds with bias.

For instance, this week it says Oakland should be favored to beat New England. Do I personally believe the perennial laughing stock Raiders are really better than the perennial A.F.C. East champion Patriots? Maybe not. But the model is telling us something to pay attention to. It’s saying: “Look deeper at the numbers that really matter. The Raiders might just be for real. The Patriots’ defense is just as vulnerable as they say.” So if you’re in an office pick ’em league and have to pick an upset, maybe this is the week to put your chips on Oakland.

An explanation of the principles behind the model and a detailed example of how it is calculated can be found here.

And now here are the game probabilities for Week 4:

Brian Burke, a former Navy pilot who has taken up the less dangerous hobby of N.F.L. statistical analysis, operates Advanced NFL Stats, a blog about football, math and human behavior.