ESPN says that they have overhauled the quarterback rating, and in typical fashion they have done it with a whole lot of hype and hyperbole. Their new innovation is called Total Quarterback Rating (which they have strangely, abbreviated as QBR).

The concept behind QBR is a sound one. Their concern is that while the traditional QB rating considers several different statistical categories, it doesn’t weigh those statistics, and it doesn’t consider when something happens in a game. For example, an interception is viewed the same in the QB rating whether it happens in a key drive early in a game or on the last pass of a half on a desperate Hail Mary attempt. The QB rating also fails to account for some things that can have a big impact on how effective a QB can be—yards gained rushing or lost after sacks, for example.

QBR works under the belief that every drive is different, and that some are more important than others. To determine how important each play is, they compare it to historical outcomes in past similar situations—same down, distance, time remaining and so on. By doing so, the QBR factors in win probability, or the amount any given play adds to or takes away from the team’s chance to win.

QBR also looks beyond the outcome of a play to see why it happened. It doesn’t just care, for example, that a pass was incomplete—it wants to know if the pass was overthrown, underthrown or dropped. On a completed pass, it also factors in the difference between different types of 20-yard passes—one that covers all the distance in the air is different than a three-yard pass followed by a 17-yard run. It also recognizes that not everything happens entirely because of the QB, so it tries to split the blame or the credit between the QB, the linemen, the receivers and so on. It also looks at how important the play is, so a clutch play is more impressive than a reasonably meaningless one.

One of the criticisms of the traditional QB rating is that it is unwieldy thanks to the maximum score of 158.3. QBR has sought to simplify the interpretation of the stat by making it out of 100. More significantly, the league average performance will be 50, so at a glance you can get a sense of how good a quarterback is. Over the course of last season, backtesting showed that a QBR of 65 indicates a Pro-Bowl caliber QB.

One interesting and potentially valuable component of the QBR is the expected values of plays. For every play they have calculated the number of points a team scores on average when they are in that exact situation. After any given play, you could subtract the expected value of the situation at the start of the play, from the situation at the end of the play to get the expected value of that play. If you add up the expected value of every play a team makes all year, you should get close to the number of points the team scores on the season—or at least that’s the theory.

There is an obvious problem with the QBR at first glance—it is very complicated. It’s not just a formula that you can plug a few stats into to get a number. Someone has to look at every single play, evaluate it, then add it all together to see what you get. It’s going to take a while for these ratings to get evaluated, and we are going to be dependent on ESPN for the preparation and maintenance of the stats.

We also aren’t going to be able to easily fully understand the numbers, so it will be hard to evaluate what is wrong with them and how they can be improved. There is probably some strong NFL handicapping potential here, but people will have to go into it with eyes wide open, to be sure that they aren’t buying into the hype and that the numbers do what they say they do. Only time will let us see how the numbers feel, and how they can effectively be used to build bankrolls with your NFL picks. Because there is so much human involvement in the creation of the stats as well, we need to become confident that there is no room for interpretation—that two people could look at the same game and get the same number. If they can’t, then there are real issues.

From an NFL betting perspective, there is one big problem with the ESPN hype machine. They have been brutally harsh about the traditional QB rating. Essentially they say that it has no redeeming value whatsoever. There’s just one problem—that really isn’t true. Passer rating can actually be a strong indicator of team success—especially when you look at the passer rating differential, or the difference between offensive and defensive passer ratings. Simply put, teams with a high differential are much better than teams with a low one.

The bottom line is this—you should be intrigued by the QBR, and you should put some time into understanding it and seeing what it can do for you, but be cautious and understand what is really happening—not what ESPN wants you to believe.