Wednesday, January 6, 2010
Plus/minus is not the devil
By Henry Abbott
Plus/minus statistics have been in the NBA for a while now. Just long enough, apparently, to start pissing some people off.
Of the many people who work with plus/minus and its derivatives (various kinds of adjusted plus/minus, Cavaliers' consultant Dan Rosenbaum's work, the Houston Rockets' voodoo, TrueHoop Stat Geek Smackdown contestant Stephen Ilardi) Wayne Winston is surely the least measured in how he talks about his powers. Like a parent discussing his child, or an engineer discussing his creation, Winston can be a bit of a zealot for his work. It makes sense that his words would stir up some strong feelings.
A fairly randomly selected example of that (I have heard this from radio hosts, e-mailers, commenters, bloggers) comes from C.A. Clark writing for the Laker blog Silver Screen and Roll. Clark notes that the plus/minus numbers in last night's Laker boxscore show some crazy stuff, like that Derek Fisher was really good, and that Lamar Odom was just OK, when watching the games reveals the opposite. Then he pokes fun at some of the metric's other findings, for instance that in his first two seasons of play, the Thunder didn't play well with Kevin Durant on the floor, or that Dirk Nowitzki is a leading candidate for this year's MVP award.
At what point do you stop trying to make adjustments to correct a flawed system and just admit that it's flawed? I'm sure adjusted plus/minus can often help to identify players who do things that don't show up in the box score. I know that defensive contributions are difficult to measure. I would imagine players like Shane Battier are more properly evaluated by APM than by traditional stats. But guess what? You can also properly evaluate those players by WATCHING THE DAMN GAMES.
Clark, I assure you, speaks for many.
And I hear what he's saying. Yes, you have to ultimately have to watch the games if you're going to be an authority on them. Basketball is still far more nuanced than the numbers. Probably always will be. But as we just learned, there are about four million seconds of NBA games in a season. Almost nobody can watch them all. You need to measure what happens in some way. Also, anyone who has ever enjoyed reading Freakonomics or Malcolm Gladwell or any of a number of similar writers can tell you that one person's gut is often not nearly as effective as actual evidence. It's just the world we live in now -- there's a ton of data out there. Are you going to ignore that reality, or get smart in making it part of your arsenal?
I think people who hate plus/minus tend to hate it for a few reasons:
- It's new, weird, and apparently here to stay, even in the box score.
- The place most people see it is in the boxscore of single games, even though plus/minus experts say one game's worth is not meaningful.
- It looks like a statistic that attempts to wrap up every little thing a player did that mattered, and it's imperfect at this.
- In the high school jocks vs. geeks debate, the geeks won the dot-com money and the Wall Street jobs, but this here is sports, pure jock territory, so it's not nice having all these geeks with spreadsheets hanging around talking smart.
My response: Chill out! Plus/minus is not invading. It's not here to take over. The women and children are safe. Plus/minus will not replace everything you ever thought you knew about sports. It's not here to extreminate your coach, your MVP voter or your eyes.
It's just one more piece of information.
Unless you're Wayne Winston or one of a small number of devotees, it's not even really a ranking of a player's total quality. Instead, it's a measure of how a team did when that player was on the court, which as I see it may or may not be because that player was on the court.
There's noise in every single part of the boxscore. Every column. Does the assists column do a good job ranking players by total quality? No, it does not. The scoring column, the rebounding column ... If you're using any one column to determine a player's total quality, that's your mistake. They will all let you down.
Including the plus/minus column. It is simply not a measure of a player's total quality. But just as it's interesting to know who got the most rebounds, or how many minutes somebody played, it's also interesting to know which players happened to be on the floor when a team played its best and worst. Sometimes it really tells you interesting things.
That's what plus/minus is, mainly: a guide to who was on the floor when the team made its good and bad runs. For instance, here's a plus/minusy chart. Look at it for a while, get comfortable with it, and you'll see something kind of cool: When the Lakers and Rockets both had their starters in on Tuesday night, the Lakers were way better. Good to know, right? Looking at points and rebounds won't really tell you that. Look again, and you'll see that when Derek Fisher sat, the Rockets went on a little run -- that's part of the reason his plus/minus was so good. Was that dumb luck on his part? Maybe Fisher's replacement, Jordan Farmar, really was the weak link. Or maybe he was just unlucky to check in before a run the Rockets were going to go on anyway. That's when I'd say go watch the video of that part of that game, and see what you learn.
Your option, as a basketball fan, is to know this plus/minus information, or not. Weird though plus/minus may seem at times, I can't imagine why you would not want to at least take a gander. Either it'll be useful or you'll ignore it. It's just a question of putting it in perspective. In general, with this stat:
- Experts almost all agree you need big samples to make plus/minus worthwhile. Not even Winston advocates one game plus/minus as a measure of a player's total contributions. Winston sometimes cites three-game measures, but acknowledges that's a bit statistically reckless, and he says that is most useful in determining things like who is playing really well and who may be hiding an injury. (The Durant case was two full seasons of his team giving up a zillion extra points when he's on the floor. Some people think even two whole seasons was not enough to say anything categorical.)
- No other measure does anything meaningful to measure a player's defense. Now, it's not always clear exactly where, but literally everything that matters to basketball is in plus/minus somewhere. Remember, the final score of every game is the total plus/minus of that game. If it matters to the final score, it matters to plus/minus. Until they measure and chart defense (it won't be easy!), plus/minus the only measure that we have to know who's playing it well. With big samples, it's worth using as a tip sheet. For instance, the Durant numbers inspired me to watch the video. The video was plain: His pick-and-roll defense, in his first two years, was not good. It's video I never would have watched without the suggestion of plus/minus that something was up -- it's also a conclusion I wouldn't have reached about his defense without actually watching the video. (It kills me, by the way, that I can't show you that video.)
Another example: On Tuesday night Portland, a slow team, played super fast Memphis. Portland's starting point guard is Andre "not as fast as I used to be" Miller. Was Miller going to get back and stop O.J. Mayo and Mike Conley on the break? Not so much, it turned out. Instead, he abandoned that task much of the game to crash the offensive boards like the center Portland doesn't have. Heroically, one of the smallest players on the floor, who prefers not to jump, grabbed a whopping ten rebounds, five at the offensive end.
But as Miller was seeking offensive rebounds, he was busy not slowing the Grizzlies in the open court.
Was that a shrewd move on Miller's part? Were the rebounds more valuable than getting back on defense? I don't know. Blazer fans certainly cheered his grit.
But Coach Nate McMillan acted less impressed. He benched Miller for almost the entire fourth quarter, only inserting him when Juwan Howard fouled out with eight seconds left. (That also could have been because Miller is one of the team's elder statesmen who had been playing really hard for two straight nights.)
So, was Miller helping the team or hurting it with his Rodmanesque ways? Would the Blazers have been better off with Miller running the team or Jerryd "fresh legs to get back on D" Bayless who played instead?
It's an honest and important basketball question, and one that's very hard to answer. But as smart people mull that, wouldn't they like to see a simple record of how the team fared with each guard running the show? If one of them was plus-three and the other was minus-one then maybe it wouldn't really move the conversation along. But what if one of them was plus-18 and the other was minus-20? Why would you not want to know that, at least as a point to investigate further? Again, not that it works as a ranking of player quality, but it's a clue about what's working.
As it happens, in this game Miller was minus-11, and Bayless was plus-seven. If you're one of those Blazer fans livid at McMillan for benching Miller in the fourth, that ought not be the final word in proving McMillan was right. Remember, this is a tiny sample size, that may well mean nothing. Could be random noise. But it ought to at least open your mind to the idea that he might have seen something. It ought to open your mind to digging in further before forming any emphatic view, or arguing too loudly that the team was blatantly better with Miller in the game.
The stats people who strike me as the best -- including those working for the Rockets, Nuggets, Celtics etc. -- are hungry for all the information they can get. The challenge is in knowing how to use that information, how much import to assign to it.
But there's not really a strong argument for ignoring the most interesting parts of plus/minus, especially when defense is half the game and every other measure essentially ignores it.
In the meantime, when you hear about plus/minus, or adjusted plus/minus, be not afraid. Yes, it's new. Yes, it sometimes says things that are counterintuitive. But it's here to stay. And as we learn how to use it, I'm certain it's going to become more and more useful.