Video data polishes College Total QBR

Wire photos

Every year, ESPN’s College Total QBR metric undergoes minor changes at the end of the season. In the interest of complete transparency, an explanation and analysis of the 2014 changes are below.

Total QBR is an all-encompassing metric that captures all aspects of a quarterback’s play – passing, rushing, sacks, fumbles, penalties, etc. It is built off play-by-play data and accounts for down, distance, field position, clock and score to determine which quarterbacks are the most and least efficient in the country. A full explanation of Total QBR can be found here.

College QBR differs from the NFL version in a few important ways. First, College QBR adjusts for the strength of opposing defenses and the NFL version does not. This is necessary in college because of the varying competition faced in conference and non-conference play.

Another important difference is that the NFL version uses live video tracking to capture data such as air yards of passes, number of pass rushers, run type, etc. This information is not widely available for all FBS schools, particularly the lower-level ones, so - during the season - this component of QBR is estimated from play-by-play data (down, distance, target position, etc.). The estimates are based on statistical analysis and modeling.

Once the season is complete, ESPN obtains video-derived data for the majority of FBS conferences (all Power 5, American, Mountain West and a few others) and replaces the estimated component of QBR with exact data.

Factoring in the exact data generally does not result in significant changes to a player’s season QBR, though there were some notable changes in 2014. A complete list of updated player QBRs can be found here, but below are some notable changes. As you will see, most changes are a result of air yards and scrambles for quarterbacks.

2014 final QBR numbers

Marcus Mariota remained No. 1 in Total QBR after the postseason adjustments. His QBR remained relatively unchanged, and he ended the season with a sizable lead over the second-place finisher, J.T. Barrett.

Top 10 in Total QBR 2014 Season

The biggest mover in the top 10 was Michigan State’s Connor Cook. Why? Sixty-three percent of Cook’s passing yards came through the air (rather than after the catch), the highest percentage of any Power 5 quarterback with at least 100 passes. In other words, Cook did not rely on yards after the catch for his passing yards.

Blake Sims fell from second to fifth in QBR for reasons similar to Cook’s rise. Led by Amari Cooper, Alabama gained 54 percent of its passing yards after the catch, the third-highest percentage in the SEC. Sims also benefited from some improbable long plays; he led the SEC with 24 completions of 30 yards or longer, yet six of those completions came on passes thrown behind the line of scrimmage (see Cooper’s 52-yard touchdown against FAU). No other Power 5 player had more 30-yard completions (six) or touchdowns (four) on passes behind the line of scrimmage than Sims.

Barrett replaced Sims as No. 2 in Total QBR. Barrett was helped by his scrambling: a Big Ten-high 315 rush yards (7.3-yard average) and three touchdowns.

Looking beyond the top 10, Clemson’s Cole Stoudt had the largest decrease in Total QBR (-7.8 points) among qualified players after the addition of tracked data. Stoudt’s average pass traveled 6.5 yards past the line of scrimmage, two yards shorter than the Power 5 average (8.7).

Conversely, UNLV’s Blake Decker had the largest increase in Total QBR (+4.8 points). His average pass traveled 10.9 yards past the line of scrimmage, two yards farther than the Power 5 average. He gained 364 of his 366 rush yards on scrambles, and a player generally receives more credit for a scramble than a designed rush in the Total QBR calculation.

Overall, the teams that relied heavily on quick, short screens (Washington State, West Virginia, Texas Tech) were negatively affected by the updated information, and the ones that passed downfield more frequently (Michigan State, Minnesota, Florida) were positively affected.

Though these changes were minor – two players had QBR changes of more than five points – adding the additional information at the end of the season made QBR more accurate by adding information that allows us to isolate the quarterback's effect on each play. Feel free to ask any questions in the comments section of this post and we will do our best to answer.