How to value Alex Smith's performance stood among the most debated subjects on the NFC West blog last season.
Smith ranked ninth in NFL passer rating among a broader group featuring Aaron Rodgers, Drew Brees, Tom Brady, Tony Romo, Matthew Stafford, Matt Schaub, Eli Manning, Matt Ryan, Ben Roethlisberger and Philip Rivers.
The 49ers' coach, Jim Harbaugh, called Smith "elite" and promoted him for the Pro Bowl. But when it came time for the 49ers to pay Smith this offseason, they gave him a three-year deal with an easy out for the team after one season. The contract bore little resemblance to the ones those other quarterbacks have commanded.
Total QBR, the metric ESPN's Analytics Team developed to more fully assess how quarterbacks contributed to winning, supported the 49ers' valuation.
Smith, despite quite a few high single-game QBR scores when playing at home, ranked 22nd among qualifying quarterbacks overall with a 46.4 score out of 100 (50 is average). His share of blame for the sacks he took hurt his team more than the sacks any other qualifying player took, a huge drain on his score.
The weekly in-season "QBR Ranks" posts on this blog provided the basis for discussions on NFC West quarterback play. We had some healthy debates over the usefulness of QBR and how it could be improved. Some of those discussions go on internally, too.
The changes will lessen the blame quarterbacks receive when they fumble during a sack, shifting more of the blame to offensive lines. Also, kneeldowns and spikes will no longer factor; those plays had very little impact on QBR over the season, but they wielded more influence on single-game scores.
Smith fumbled seven times and lost two of them. His fumbles were not particularly costly overall, allowing Smith to rank ninth in fewest expected points lost to fumbles. Brees was first. Tim Tebow was last.
These QBR tweaks were relatively minor. The Analytics Team discussed other possibilities at the most recent Sloan Sports Conference.
"One of the things that does sit a little bit on my mind is that we fundamentally have to do it on a per-play basis because we're going to be looking at how well did they play on third down vs. second down vs. five or more rushers and these are great," Oliver said recently at the conference. "One of the things I wonder about is whether that is the right basis for evaluating a quarterback overall."
The current system assigns greater value to scoring drives requiring fewer plays, all else equal, on the theory that scoring quickly would be more impressive than if finding the end zone took longer.
"We talked about some sort of QBR per drive, because if you go 80 yards in three plays vs. 80 yards in 12 plays, why should the three-play drive be four times better than the 12-play drive?" Oliver said. "In many cases, the 12-play drive is better. I don't know how we do that, but it is something we have talked about.
"For most of the work that we do, that doesn't affect anything, but I think it's a great conceptual question that hopefully we can figure out in the near future."
I found QBR most useful when it diverged significantly from NFL passer rating, as it did notably for Smith. Using the formula to declare one quarterback absolutely better than another made little sense. But if we could find out why QBR diverged from NFL passer rating or our perceptions in general, that could be of value.
For Smith, taking sacks spelled a large part of the discrepancy. Some made the case that Smith's offensive line was disproportionately responsible for many of those sacks. I thought Smith was content taking sacks to avoid interceptions, a tradeoff that helped explain the gap between NFL passer rating, which does not account for sacks, and QBR, which does.
My current take: Offensive lines are more to blame for some sacks, perhaps explaining why a QBR score suffered unexpectedly for a single game. Overall, though, the blame distribution evens out, creating more reliable results for a full season.
This discussion isn't for everyone. Apologies to those who don't care for analytics as they relate to football. My hope is to find more relevant applications.