Adjusting GAA for Shots Against

Over my last few posts I’ve been looking into the idea of game level goaltending performances and adjusting goalie evaluation metrics to account for the number of shots against a goalie faces. One of the issues I have with Save Percentage as a method of evaluating goaltenders is that it has the potential to overstate the value of goaltenders who tend to face a lot of shots, or understate the value of a performance when a goalie faces few shots.

To give an example, let’s look at 10 hypothetical games for 2 (obviously hypothetical) goalies, which I’ve set up in the table below. Over the 10 games, each goalie faced the same number of shots and let in the same number of goals (i.e. identical save percentages) but did so in different ways. Goalie A was consistent throughout all the games, while Goalie B performed very well in 3 games, decently enough in four, and very poorly in 3 other games.

I think most people would agree that they’d rather have Goalie A over Goalie B, and while this is an extreme example, I think it highlights a flaw in Save Percentage as a metric and why more granular game based analysis can be necessary. The three very good games put up by Goalie B shouldn’t cancel out the 3 very bad games.

I’m hesitant to bring up goals against average as a “solution” to this problem, but bear with me for a second. Preventing goals is a goalie’s job, so GAA seems like it should be a natural measure of a goalie’s ability. But GAA has several flaws that really make it a bad metric, primarily that it doesn’t take into account the number of shots against. A goalie who faces 50 shots and lets in 3 goals is given just as much credit as a goalie who faces 20 shots and lets in 3 goals. Clearly if we could adjust GAA to incorporate the number of shots allowed GAA would be a more useful metric. The question then is how we should properly adjust GAA to evaluate a goalie’s performance in a given game.

The chart below shows the average goals allowed plotted against the number of shots against for all games since 2007. What we see intuitively makes sense: as the number of shots faced increases so does the number of goals, but the trend is more logarithmic than linear. If save percentage increases with shots against, we wouldn’t expect goals allowed to double just because shots against doubled.

Average GA Per Game vs. Shots Against

With this in mind, we can create a model to evaluate how goalies performed on a game by game basis vs. the average goaltender facing the same number of shots. Taking all goaltending performances since 2007, we can look at how many goals we would expect an average goalie to give up, given the amount of shots against. Then, if we look at how many goals a given goalie gave up in each game and compare it to the expected goals against set by our model, we can come up with a shot adjusted goals against metric that provides an estimate of how many additional goals a goalie prevented (or allowed) over the course of a season. I’ll define two new metrics Goals Against Minus (GA-) and Goals Against Average Minus (GAA-) as:

GA- = ∑ GA – E(GA)

GAA- = (∑ GA – E(GA))/N

where GA is the goals allowed in given game, E(GA) is the Expected Goals Against given by the model, and N is the number of games played (ideally this would be the number of minutes played divided by 60, but I haven’t gotten around to setting that up yet).

This new metric has a few nice properties that adjust for what I perceive to be some of the problems with Save Percentage or Goals Against Average as metrics. First, for a constant goals against, a goalie who faces more shots against is rewarded more than a goalie who faces fewer shots. Second, for a constant save percentage, goalies who allow fewer goals against are rewarded more than goalies who allow more goals. And finally, it measures a goalie’s performance on a game-by-game basis, rather than overemphasizing certain games based on the number of shots faced.

The tables below show the top 10 and bottom 10 goalie performances in GAA- for all years since 2007 (for goalies who played a minimum of 33% of the season).