Thursday, April 10, 2014
Inside Slant: Projections based on college
By Kevin Seifert
(Another in an Inside Slant series that will appear regularly during the 2014 offseason.)
The accolades pop off the page. In many ways, Georgia's Aaron Murray is the most prolific passer in the history of the SEC. No quarterback has completed more passes for more yards or touchdowns, and Murray is the only quarterback in conference history to throw for at least 3,000 yards in four consecutive seasons.
As the NFL draft approaches, however, Murray is not viewed as a top prospect. His success in the conference best linked to NFL-level play has been trumped by concerns about his size and, temporarily, his recovery from a torn ACL. ESPN's Scouts Inc. rates him a fifth-round prospect , citing his measurements at the February scouting combine -- just over 6 feet, 207 pounds and with 9[-inch hands -- as impediments to throwing from an NFL pocket.
Murray represents the 2014 embodiment of an annual draft debate. What is more predictive of NFL success: college production or projected athletic ability, as manifested in combine measurements?
There are countless anecdotal illustrations of this argument, ranging from the infamous Mike Mamula -- a 1995 combine star whose football skills were more limited -- to Clay Matthews, who produced twice as many sacks as an NFL rookie in 2009 than he did in four seasons at USC. Recently, a group of college professors worked to inject some hard numbers into the discussion via a study of 640 drafted prospects over a three-year period from 2002-04.
Their results were instructive. College production, averaged per game and scaled based on competition level, was at least twice and in some cases three times more indicative of NFL success than athletic ability. In fact, said Georgia professor Brian J. Hoffman, combine numbers added nothing to the accuracy of projections that college production hadn't already accounted for.
"If it were up to me," Hoffman said, "I would certainly [tell general managers] to ignore the combine. Completely ignore the combine. My concern is that, if anything, it leads you astray more often than helps bring you a good player. There are some exceptions, particularly with a player like [New Orleans Saints tight end] Jimmy Graham, who played only one season and so you have less data. But focusing on college performance seems a much more reliable approach. In general, college performance will tell you what you need to know."
This should be no surprise in the business world, where past performance and experience are far more valued than aptitude tests and measurements. In professions requiring physical skills, of course, hiring managers feel compelled to project aptitude. The combine is the primary NFL vehicle for that task, but this study suggests its results are at best redundant.
At the same time, it's important to acknowledge the limitations of this data, which can be viewed in detail here. It doesn't ensure a player will be successful if he put up big numbers at a BCS school, nor does it mean he will flop if he didn't.
Matthews, who managed 5.5 sacks in his college career, is the perfect example. The Green Bay Packers put faith in their physical evaluation of Matthews, as well as their analysis of his play even when he didn't record sacks or tackles, and were rewarded with a pass-rusher who is on a Hall of Fame track (50 sacks in 69 games).
In sum, the study showed that the statistical correlation between college production and NFL success is 0.3, which is about the same as the correlation between high school grade-point averages and college grade-point averages. In other words, NFL teams have plenty of additional analysis to complete beyond college production.
In any event, there are some important thoughts to be gleaned here. First, if the data compiled via the combine's athletic measurements has proved statistically worthless, it seems time to reconsider the nature and substance of the drills. The results, as Hoffman said, are more likely to cause a draft mistake than contribute to a successful choice.
Second, it is another reminder to look closer at players like Murray. Hoffman noted the inherent bias of working at the school Murray played for, but the study suggests Murray's production merits more weight than NFL teams typically assign.
"I don't think this tells us absolutely that a player will do it in the NFL if he's done it in college," Hoffman said. "But it also doesn't make a lot of sense to say they probably can't do it in the league consistently based on these physical measurements. There are always going to be exceptions, but when you look at a guy like Aaron Murray or Drew Brees, so-called undersized guys who were shattering records in college while playing in a pro-style offense, you look at the data and suspect he would have a better chance to succeed than NFL teams might think."