Baseball Tonight continues its look at statistical analysis by looking at pitching evaluation methods. This piece takes a closer look at evaluating starting pitching
A few years ago, ESPN baseball columnist Rob Neyer wrote a piece about why the quality start is actually a quality statistic. The key argument in his article is that in 2005 there were 2,447 quality starts in the majors, and a team won 67.4% of those starts.
This got us thinking…what if we could somehow predict the winning percentage of ANY team given ANY starting pitching line. This seems like it would improve the quality start metric, as it would give a better indication of how a pitcher impacted his team’s chance of winning. Others, such as Bill James, have devised methods to do something similar (you may be familiar with "Game Score," listed in ESPN.com box scores). We tried another approach.
First, we compiled the starting pitching lines from every starter for the last five seasons. There are approximately 2430 MLB games per season, two starters per game, for five seasons (2005-2009). That's 24,300 observations.
We then used a statistical technique known as binary, or logistic, regression to predict a team's probability of winning based on the starter’s pitching line. Essentially, we plug basic stats from a box score (such as innings pitched, earned runs, walks, strikeouts) into a mathematical model and the model then spits out the team’s chance of winning the game based on those stats.
In order to simplify the model and keep it as close to the current criteria of a quality start, we used only innings pitched and earned runs as variable for the regression. It’s also worth noting that those two stats – IP and ER – also had the largest statistical impact on a team’s win probability.
Let’s get to the data: Here’s the predicted team winning percentage for a few different combinations of IP and ER by the starting pitcher:
Pitcher A: 6 IP, 3 ER Team win pct = 49.6%
Pitcher B: 7 IP, 3 ER Team win pct = 55.0%
Pitcher C: 9 IP, 4 ER Team win pct = 54.5%
Pitcher D: 6 IP, 2 ER Team win pct = 60.9%
What’s most interesting here is that we see at least one non-quality start stat line (9 IP, 4 ER) which gives the team a better chance of winning than the minimum quality start criteria of 6 IP and 3 ER.
As to the question of how we can possibly improve the existing quality start metric. Here is a quick example of how our model helps better judge which starting pitchers truly impacted their team’s chance of winning the game.
Using the current definition of quality starts, Roy Halladay and Doug Davis tied for 15th in the majors last season with 22 quality starts. We think most fans would agree that Halladay is arguably a better pitcher than Davis and likely helped his team win more games.
That’s where our new model can help.
If we set the our quality start threshold to any start where the starting pitcher gave his team at least a 75 percent chance of winning, Halladay had 13, nearly twice as many as the seven by Davis. Halladay’s 13 tied for sixth-most in the majors. Here’s a look at the top five from last season:
A HIGHER QUALITY
A list of the pitchers with the most 2009 starts in which their team had a 75+ percent chance of winning by this method
Now, let’s remember that this is just a start (no pun intended), as this regression model can certainly be improved by adding more variables and conducting further tests. Hopefully, though, this a good primer for a different way to judge starting pitching success, and will spark some interesting discussions in ballparks and bars across the country this season.