NFL Draft 2.0
There is a new test that hopes to lessen the guesswork of NFL player drafting.
Among all the potentially humiliating measuring sticks that draft-eligible college players will be subjected to this week at the NFL's annual talent meat market known as the Indianapolis scouting combine, many players hold a special contempt for the Wonderlic test, which is supposed to measure their intelligence.
When current Redskins cornerback DeAngelo Hall was shown a 1930s-era photo of the real Dr. Eldon Wonderlic after taking the test, Hall -- who was on camera -- leaped up and said "Gimme that!" and snatched the photo from a reporter's hand. Then Hall made a comical show of punching the photo two, three, four times while laughing as if revenge was sweet.
Current Bears wide receiver Roy Williams' response was more existential: "Why, Mr. Wonderlic? Why?"
Football players take a lot of, well, crap -- that's the word -- when it comes to the idea that it requires much intelligence to do what they do.
There is widespread acknowledgement that the NFL game itself is more sophisticated and strategy-driven now than ever before. But, paradoxically enough, the players still endure everything from gentle ribbing to insults to stinging condescension when it comes to how much they themselves actually have going on between the ears.
But one change to this year's combine raises an interesting question about all that: What if NFL teams weren't looking in the right places all along? What if the NFL hasn't been assessing intelligence in the most optimal ways when it comes to predicting what makes a successful NFL player?
That's the animating idea behind the Player Assessment Tool (PAT). Cyrus Mehri, an employment lawyer, and Harold Goldstein, an associate professor of industrial/organizational psychology at Baruch College, City University of New York, devised the test over the past two years with the help of the NFL's General Managers Advisory Committee. They've persuaded the NFL to try it, starting this year.
"What we're trying to capture is the different ways that people are smart," Mehri said in a phone interview this week.
It's a fascinating challenge. Nobody knows exactly why some physically gifted football players fail spectacularly while less-gifted players excel, or what constellation of traits teams should look for to better predict success, especially when it comes to quarterbacks. But NFL teams devote a ton of time, effort and money to figuring it out through vehicles like the combine. Many teams already use other assessment tests in addition to the Wonderlic. Yet big hits and misses still happen.
A combine warrior like former Jets No. 1 pick Vernon Gholston flops while a Danny Woodhead falls through the cracks and goes undrafted. The overall result is reminiscent of a line from that old Lily Tomlin and Jane Wagner play "The Search for Signs of Intelligent Life in the Universe": "What is reality, anyway? Just a collective hunch."
Goldstein and Mehri, a co-founder of the Fritz Pollard Alliance who helped the NFL devise the Rooney Rule, tackled the challenge by beginning with a few simple-sounding premises. One of them is the idea that the test should be specifically designed for football players, which the Wonderlic is not, and what the players actually do.
The Wonderlic is a one-size-fits-all test that is taken annually by about 2.6 million people in a variety of industries worldwide. NFL teams began administering it to players in the 1970s, following the Dallas Cowboys' lead. To them, part of the beauty of the test was its brevity. The 50-question multiple-choice exam must be completed in 12 minutes. And anyone can administer it to a player -- a team exec, a scout in the field -- and tally up an instant numeric score. (For some sample questions, click here.)
The PAT test is different, and not just because it requires players to answer more than 100 questions in an hour. The memo the NFL sent to all 32 clubs five days before this week's combine explained that Goldstein is "an expert in industrial psychology who has designed employment tests in a variety of other industries" and that he worked closely with Mehri to develop "a new tool that measures a wide range of competencies, including learning styles, motivation, decision-making skills, responding to pressure or unexpected stimuli, and core intellect."
Mehri says: "What this test is trying to get at is, What are the characteristics of success that are outside the physical domain -- the nonphysical characteristics of success? I think the assessments provided will fall under three major umbrellas. One is motivation: What motivates a person, how intrinsically motivated are they, how passionate are they? Second: different learning styles and forms of intelligence. The third one is more psychological: mental toughness, handling pressure, being a reliable teammate. Things like that."
The Wonderlic has long been criticized as having racial or socioeconomic biases. And Mehri maintains, "This new test kind of levels the playing field from a socioeconomic point of view too."
If that sounds like a lot of hocus-pocus or wishful thinking -- even an oversell -- know that retired NFL executive Ernie Accorsi was prepared to be skeptical too.
Mehri and Goldstein were sent by the NFL to pitch the new test to Accorsi first, because he now works as a consultant to the league and he's co-chairman of the General Manager's Advisory Committee. Before that, Accorsi spent decades as a GM for the Baltimore Colts, Cleveland Browns and, most recently, the New York Giants. His preference for Eli Manning over Philip Rivers in the 2004 draft -- no easy call at the time -- was a major factor in the Giants' Super Bowl victory in 2007. And he has always felt the Wonderlic test had actually served the NFL "pretty well" all these years.
Accorsi hastens to add that like a lot of his peers, he doesn't see the Wonderlic as a be-all and end-all when it comes to identifying which players to draft. "How determinative it is depends on the club," Accorsi says, "but it's usually not 'the' determinative factor."
Still, Accorsi did regard it as a useful tool, especially for positions like center, middle linebacker and quarterback. And he has no trouble recalling instances where the Wonderlic (like the additional assessment tests his Giants teams administered) were helpful to him.
"I'm not going to name names at this point," he says, "but in my time [preparing for drafts], I did personally look at the [Wonderlic] test results. And I personally gave the test to quarterbacks. I gave it to one guy, and this quarterback didn't score high at all. I was surprised. So I gave it to him again and -- surprise -- he scored even lower, which suggested to me he was just guessing. And then I thought: Wait. You know what? That's the way he played! In college, that's the way he played!
"He was mistake-prone under pressure. So to me, what these tests do the best is to confirm what you see. Sometimes you look at something and you're confused, you're not sure one way or another, and you see a test result like that and think: This nails it. So that's why!"
One of the things Accorsi likes about Mehri and Goldstein's approach, in addition to the "knowledge vs. intelligence" distinction, is another simple premise: If you have a tool that isn't great at predicting results, perhaps you need to get yourself a better predictor. Part of their pitch for the PAT was this visual: Their new test could give the NFL a better picture of what's going on inside players heads, much the way MRIs changed the league's ability to diagnose injuries better than cruder X-rays do.
Accorsi was sold enough to proceed. And when he presented and endorsed the new test on a conference call with Mehri and Goldstein to rest of NFL's General Manager Advisory Committee -- "I'm an old-school guy, I've been around longer than any of 'em, and what I said was, Of all of you, I'm going to be the toughest to convince on this. And I was still really impressed" -- the seven other GMs were all receptive too.
"All of them volunteered to participate in answering questions we used to devise the test," Mehri says.
But the NFL being the NFL, this did lead to some unintentionally funny consequences.
When Mehri and Goldman canvassed GMs like Thomas Dimitroff, John Elway and Jerry Reese, starting with the simple question of "What makes a successful NFL player?" they got instant insight into how hotly competitive teams are and how jealously secrets are guarded in the NFL. None of the active GMs wanted his counterparts to be told what their particular recipe for profiling successful players and executing good drafts is.
(Even Accorsi, who has been retired six years now, demurred when I asked him this week exactly what he told Mehri and Goldstein. "I'd rather not say," he explained, "because I don't know how much of our system is still in place with the Giants.")
Laughing now, Mehri admits: "I think there's a mixed emotion about this. One emotion is, 'Thank god, this change is something we desperately need,' which makes you wonder what took so long. The other emotion is: 'Oh my god, 31 other teams are getting it too? I want this just for ME.' ... But I have to say, the GMs could not have been more impressive. They've given this a lot of thought. And it was fascinating. Every one of them had a distinct philosophy. Seven different ways to get to the same destination."
The NFL has come a long way from the time when Pat McInally -- a punter and wide receiver from Harvard who is still the only person confirmed to have scored a perfect 50 on the Wonderlic -- was told by a prominent NFL personnel man that he actually dropped to the fifth round of the 1976 draft. The reason? Some teams equated being "too smart" with being a smartass, thus feared he was potentially "uncoachable."
When current Jets quarterback Greg McElroy, a Rhodes Scholar finalist who led Alabama to the 2009 BCS national championship, was told he scored a 48 or 43 on the test at the 2011 combine -- "I still don't know to this day what my score actually was," he laughs -- he didn't notice a tick up or down in the clubs interested in him before he was taken in the seventh round of the draft.
McElroy laughs and says he is not the sort of man who would punch a photo of Wonderlic, the way Hall did, if given the chance. He says he understands why the NFL puts players through so much scrutiny at the combine, given the million-dollar bets teams make on them. But he admits the entire experience -- not just taking the Wonderlic -- does create tension during the three days players are there.
Chuckling again now, McElroy says: "I remember going in there and seeing the test all laid out. It felt like you're taking the SAT again. I think everybody was pretty anxious. It's all part of the combine, but it's a difficult time period. You're getting measured and tested in every way possible. You have maybe eight different stations with doctors examining you for past injuries and talking into a little microphone. I think one of the other quarterbacks that I was there with, I just can't remember who, described it best. He said, 'The combine is the best experience that you never want to have again.'"
The hope is the PAT will be more of a win-win situation. Teams get a better, fairer tool to judge players. Players benefit because their differences are better understood and not necessarily seen as a bad thing. And all of that comes with the bonus of eliminating two other annual irritants that shadow the Wonderlic's use: Agents can't prep players for the new test the way they now do for the Wonderlic, which teams dislike. And PAT scores can't be leaked the way the Wonderlic scores are (often to the great humiliation of players who score abysmally low on the 1-50 scale).
The PAT purposely does not give a numeric score. Rather, Mehri says, Goldstein will "grade" each test and every team will receive one page of "coaching points" concerning each combine player to help evaluate strengths and weaknesses in different aptitude and psychological categories, as well as determine whether players will mesh with certain coaching styles. (For example, does a player pick up the playbook best with a visual aid, in writing or with an on-the-field demonstration?
Mehri goes so far as to theorize the PAT might even help better pinpoint what it takes to play quarterback -- notoriously the hardest position for NFL scouts and GMs to project.
Mehri believes with the new test, "someone like [Seattle rookie sensation] Russell Wilson will not slip to third round. He'd be a first-round pick if came out in this year's draft."
Is that another oversell? Or the truth?
"We won't know about this new test for at least a few years," Accorsi predicts.
At the very least, perhaps the idea that football players are all knuckle-draggers could and should be in for reassessment, thanks to the PAT.
Your average wideout or nose tackle may not be able to tell you the theme of Shakespeare's "Macbeth." But if you ask them what happens if X, Y and Z unfold before or after the snap of the ball, they can tick off the complicated answer.
Yes it's "only" football. But NFL players are highly sophisticated at the highly specialized things they do, and the ones who stick in the hotly competitive league are world-class at what they do. At least give them that.
Longtime offensive coordinator Al Saunders was renowned for his 800-page playbook. The NFL has come a long way from the days of vanilla 4-3 defenses and no nickelbacks.
"You have to be smart to survive today, it's a different world," Accorsi says. "I mean, the coaches all hold up what I call these Denny's menus to their face on the sideline. And the quarterbacks -- the first time I saw all this pointing before the snap, I said, What's with that? I was told, 'Well, they have to identify the middle linebacker.' I said, Hey, when I got into league, nobody had a problem identifying Dick Butkus: he was right there lined up in the same place every play, scaring the hell out of everybody."
So what makes a successful NFL player today?
Something Mehri said in passing may actually get close to that elusive definition: "They're people making quick decisions under duress with split-second timing -- all while processing a lot of information."
Nobody argues that playing in the NFL is rocket science. But NFL players do have every right to shoot back that sophisticated scientists haven't exactly been able to figure them out yet either. So there. And that makes that question Roy Williams asked at the top of this article exactly right.
Why, Mr. Wonderlic? Why?