Context Neutral Player Evaluation: Examining Defence and Calculating xGD20

A few months ago I wrote about xGF20, my attempt to isolate a player’s offensive ability from his teammates abilities and the luck involved in shooting percentages in small samples. At its core, xGF20 is based on 3 reasonably repeatable individual measures:

1) A player’s rate of individual shot generation;

2) A player’s rate of altruistic shot generation (that is, the shots he generates for his teammates); and

3) A player’s individual shooting ability (regressed based on position)

Using these three metrics, xGF20 presents a player’s expected on-ice goals for per 20 minutes of ice-time, assuming his teammates shoot at a league-average rate. While the calculation is slightly more complicated than what I’ve just presented, it does allow us to make comparisons between players without worrying about who their linemates were. If you put Jason Spezza on a line between my brother and I (sorry Ben), you’d expect his on-ice GF20 to drop dramatically through no fault of his own. Although xGF20 is very similar in theory to Relative CF20, taking into account a players own shooting ability allows us to differentiate between those who just throw the puck at the net without purpose (David Clarkson) and those who take more shots because they’re better at it (Alex Ovechkin).

Looking at xGF20 numbers over the past few years was encouraging not only because the results agreed with common intuition (Sidney Crosby had 5 of the top 10 seasons over the past 5 years), but also because it was highly repeatable year-over-year. When we calculate a player’s xGF20 for a given year, it has a strong correlation to his xGF20 in subsequent years, a good sign that we’re isolating an individual ability.

xGF20 did have a major fault as a player evaluation tool though: it only looked at one side of the game. To get a full view of a player’s contribution to his team we need to consider the defensive side-of-things as well. This is, unfortunately enough, where things get tricky. Isolating individual defensive ability is extremely hard to do, as defensive results are influenced by the efforts of each player on the ice as well as any effects a coach might have through his choice of defensive system (see here, for example, where I found that teams had a fairly strong ability to control whether opposing forwards or defencemen were taking the shots against their net).

A lot of effort in the analytics community recently has been put into finding effective ways to evaluate defensemen (who tend to, intuitively at least, shoulder most of the defensive responsibility): Tyler Dellow has looked at using CorsiRel (the difference between a team’s CF% with a player on the ice and off the ice) and found that most “famous” defensemen tend to outperform their teams results, while Garret Hohl looked at defensemen’s effect on both their team’s and opponent’s Corsi %.

We’re going to take a similar approach here on the defensive side of things, but rather than look at Corsi %, we’ll use CA20 WOWY. In an ideal world we’d be able to break down defensive contribution further, but given that there’s a lack of clear evidence to support the idea that on-ice save percentage is repeatable, CA20 WOWY gives us a decent way to measure a player’s shot prevention ability relative to his teammates. Our expected goals against metric simply takes a player’s CA20 WOWY and adds it to the league average CA20 (17.5) to calculate his expected CA20. We then multiply by the league average Corsi shooting percentage to find his xGA20:

xGA20 = (17.5 + CA20 – TMCA20) * lgCSh%

xGA20 shows significantly less repeatability than xGF20, which we would expect given that it attempts to measure something that’s far stronger at the team than individual level. If we look at the year to year correlations in the graph below, we see that that as more years pass xGA20 shows stronger repeatability than unadjusted CA20 for defencemen (interestingly, for forwards raw CA20 seems to show a stronger correlation). While the repeatability isn’t what we’d hope for in an individual stat, it does give us a starting point in measuring a player’s defensive contribution.

xGA20 Year-Over-Year Correlations

One thing that’s important to note is that we see a lot less variance in xGA20 than in xGF20 since we don’t use an individualized save percentage. While this may unfairly penalize a small subset of players who do have an effect on their on-ice save percentage, it allows us to sidestep the difficult issue of trying to seperate out a player’s contribution to on-ice save percentage from his goalie.

Now that we’ve taken care of the defensive side of things, let’s look at pulling everything together into one aggregate metric. After all, what we ultimately care about is whether or not a player is going to help our team outscore our opponents or not. To measure that all we have to look at is the difference between each player’s xGF20 and his xGA20, which we’ll call xGD20.

xGD20 = xGF20 – xGA20

xGD20 shows reasonably good repeatability, with a year to year correlation forbl forwards of 0.62 between subsequent seasons dropping down to 0.51 when we look at comparing one year with 4 years in the future. For defensemen, the correlations are lower as we’d expect, ranging from between 0.41 (year y to y+1) to 0.47 (year y to y+4). What’s important to note though is that it tends to be more repeatable than CF% WOWY for forwards, and at the same level for defencemen (to be expected since the defensive shooting is heavily regressed and they take fewer shots themselves).

xGD20 Year-Over-Year Correlations

From a descriptive statistics point of view, the mean for both forwards and defencemen are right where we’d expect them to be around 0. We again see more variance in the forward’s numbers, driven by the higher variance in regressed shooting percentage.

One of the interesting things that I’ve found is that xGD20 seems to relate fairly well to even-strength ice-time. If we break up the list of forwards from 2013-2014 into 4 groups by ice-time, we see that xGD20 decreases steadily by each group.

We also see the same result, albeit to a lesser degree, if we break up the defencemen by ice-time, with first pairing blueliners showing better xGD20 numbers then 2^nd and 3^rd pairing players respectively.

Looking at the top 10 forwards from last year we see a lot of expected names, as well as a few players who appear to have had more extremely unlucky seasons. Sidney Crosby unsurprisingly leads the list, while Joe Thornton, Patrice Bergeron, Corey Perry and Jonathan Toews aren’t shocking to see at the top. In contrast, while Alexander Semin and Alex Burrows both had disappointing years, their past shooting ability and their shot numbers last year suggest that they’re due to bounce back next year.

On the blueline, TJ Brodie and Mark Giordano lead the way, although a large part of that is driven by their amazing xGA20 numbers (which are in turn related to Calgary’s awful defence). Upcoming free-agents Matt Niskanen and Anton Stralman look like they could be good value pickups, with their xGD20 rankings outpacing their traditional stats. Interestingly, most of the players made this list by holding their opponents to fewer shots, rather than through their own offensive ability – all of the top 10 defencemen are well below in xGA20, while only one of the top 10 defencemen in xGF20 made the list (Lubomir Visnovsky).

I’ve uploaded a full list of xGD20 for all players in 2013-2014 to a Google Spreadsheet here. If you’re looking for a primer on who your team should avoid come July 1^st, the bottom of the list will certainly come in handy.

Obviously there are still improvements to be made-the list of top defencemen above highlights that players on extreme teams can observe extreme results in the short term. If we look over a longer horizon though, xGD20 does seem to provide a fairly consistent view of a player’s talent, and one that intuitively makes sense. All it attempts to measure are things that we know a player has a degree of control over-how many shots he takes, how many shots he sets up for his teammates, how good of a shooter he is, and how well he prevents shots against his own net. Everything else is, generally speaking, too variable to use to evaluate players, and as such we need to take it out of a player’s results to get a true sense of his talent independent of the results in any given year.