Examining The Drivers of Score Effects I: Shooting and Save Percentage

Score effects – they’re real, and they’re spectacular. And while the idea and impact of score effects are generally understood by most in the hockey analytics community, they can often be a difficult subject to introduce to newcomers to the field. A solid knowledge of score effects is critical to understanding why teams that outshoot their opponents in a given game tend to lose more often than not, or why a player or team’s unadjusted statistics may be misleading you.

Furthermore, while many people have hypothesized about what causes score effects, there’s currently little documented analysis examining the actual drivers of score effects. So consider this article, and its coming sequel, as first steps at drilling deeper into the factors that lead us to observe score effects. Hopefully these pieces will serve as both a reference point to provide a basic understanding of score effects, and as evidence that helps justify any adjustments we do make when devising new statistics.

What are score effects?

Score effects generally refer to two changes that happen in a hockey game depending on the current score. When a team is up by 1 or 2, our expectation of how they’ll perform (and consequently what stats we’ll record) is different than when they’re down 1 or 2. These changes often have a non-trivial impact on the stats teams and players accumulate in any given game or season, and consequently can change our impression of them fairly significantly.

The first change, which is generally more widely discussed, is that teams that are trailing tend to take a greater percentage of the shot attempts, while teams that are leading (obviously) take a lower percentage of the shot attempts.

Corsi For Percentage by Score State

Corsi For Percentage by Score State

This is the factor that lead to the development of Score Adjusted Corsi and Fenwick, first popularized by Eric Tulsky, and more recently updated by Micah Blake McCurdy. Score Adjusted Metrics have been shown to better predict future results than non-Score Adjusted Metrics, highlighting the impact that score effects can have on our understanding of a team’s underlying talent.

The second change, which is referenced far less often, is that teams that are leading tend to score on a higher proportion of their shots (i.e., they post a higher shooting percentage) than teams that are trailing. The obvious corollary to this is that teams that are leading also see their save percentage rise, giving them a percentage advantage on both offense and defense. And while we’ve known for a while that these changes happen, there’s been less research on why the percentages change when the score changes, and if there’s any information in these changes that we can use to better understand the performance of a team. It’s the drivers of these percentage changes that we’ll look into in this article, to see if we can figure out what’s really behind the shifts we see.

Wait, let’s back up a step first – do teams in the lead really score on more of their shots and allow fewer of their opponents’ shots in?

Yes, as unbelievable as it may sound, teams become better shooters and goaltenders when they’re in the lead compared to when they’re trailing.

Shooting Percentage by Score State

Shooting Percentage by Score State

Save Percentage by Score State

Save Percentage by Score State

There are 2 things to note here: first, when games are tied teams play better defensively, and fewer shots go in than when one team is leading. This is true for both offense and defense, and doesn’t necessarily have profound implications for our analyses, but is nonetheless interesting to note.

Second, goalies and shooters for teams who are leading get a significant bump when they’re up a goal or two: shooters who are up by 1 tend to score on 0.6% more of their shots than shooters who are down by 1, and shooters who are up by 2 nearly hit a 9.5% shooting percentage, almost 2% higher than they shoot when the game is tied. Goalies see the same trend: their save percentage when up is 0.5%-1.5% higher when leading than it is when they’re behind.

 

So why do teams shoot better when they’re leading? Is it because the teams with better shooters are generally leading more often?

Actually, no. To test this idea we can bucket each team by their shooting percentage in tied games and see if the general trend holds. The tied shooting percentage should give us a reasonable estimation of a team’s true shooting talent, so if we still see score effects after we’ve bucketed these teams it’s reasonable to conclude that it’s not an issue of better shooting teams playing with a lead more frequently.

Shooting Percentage by Score State and Tied Sh%

Shooting Percentage by Score State and Tied Sh%

And when we look at the data, that’s exactly what we see – at each level of “natural” shooting talent, we see the same trend, with the shooting percentage when leading landing higher than the shooting percentage when trailing. There’s clearly more to it than just talent differences, as the effect tends to persist across different types of teams.

Ok, what about on defense? Maybe the teams that have better overall goaltending are in the lead more often?

Once again, the answer is no. If we do the same analysis we just did, but this time we group by save percentage, we still see score effects in each group. Even for the teams whose goalies posted a sub-0.910 save percentage when tied, they still received better goaltending when in the lead than when behind. Once again, we see that the effect persists even after we take into account any differences in underlying ability.

Save Percentage by Score State and Tied Sv%

Save Percentage by Score State and Tied Sv%

Is it where teams are shooting from? Are good teams just getting more shots from high danger areas?

This seems to be partially the case – when teams are down or tied, the distribution of their shots in generally the same, with the % of shots taken from the high danger zone unchanged between the -2, -1 and tied score states. There is some shift to taking more low danger shots and fewer medium danger shots when losing, but it’s not a significant move.

  % of Shots From Danger Zone
Score Differential Low Medium High
-2 45.3% 27.1% 27.6%
-1 45.0% 27.4% 27.6%
0 44.9% 27.6% 27.6%
1 42.9% 28.7% 28.5%
2 41.8% 28.6% 29.6%

We see the real differences when teams are ahead, as those clubs take a larger portion of their shots from the higher danger zone (29.6% when up 2 vs. 27.6% when tied) and a lower amount from the low danger zone (41.8% when up 2 vs 44.9% when tied). Given that high danger shots are (obviously) more dangerous, can we conclude that this shift alone is enough to explain the changes in shooting and save percentage that we’ve noted above?

To answer that question, we need to look at whether score effects exist within each danger zone; in other words we need to know whether a player who takes a shot from the slot while he’s ahead 2 is more likely to score than a player who takes a shot from the same spot in a tie game.

Danger Zone Shooting Percentage by Score State

Danger Zone Shooting Percentage by Score State

And as it turns out, teams do actually shoot better in each of the three danger zones when they’re leading (and similarly, they shoot worse when they’re behind regardless of shot location). So while the shift towards more high danger shots when leading might explain some of the increase in shooting percentage, it’s clear that there’s more to the story than just that. It may be that teams that are leading are taking more of their high percentage shots on odd man rushes, or that defenders are more frequently out of position as they pursue more aggressively in an attempt to regain possession – in any case, teams are able to convert more frequently no matter where they are on the ice if they’re ahead.

This is critical to know as we move towards the development of expected goal models, as it indicates that the score still has a major impact on the likelihood that a shot goes in. Knowing the shot location is clearly important, but adding in the score state gives us more information that can further refine our assessments, and help us form a better understanding of a team or player’s talents.

That’s a lot to take in. Can you summarize it all for me?

Sure, basically what we’ve found is:

  • Teams post higher shooting percentages when they’re ahead than when they’re behind. But they also see the lowest shooting percentage in tie games.
  • Consequently, teams post higher save percentages when they’re ahead than when they’re behind. When they’re tied, they record the highest save percentages of any score state.
  • The increase in shooting percentage when leading happens to teams regardless of how much underlying shooting talent they have.
  • The increase in save percentage when leading happens to teams regardless of how much underlying goaltending talent they have.
  • Teams tend to take more of their shots from high danger zones when they’re ahead. Teams tend to take more of their shots from low danger zones when they’re behind.
  • Score effects happen within danger zones as well – even after we’ve split up the shots by danger zone, we still see the same patterns that we do in the overall dataset. In other words, the shift in where shots are taken from is not enough to explain why teams shoot better when they’re leading.

Wait, what about the whole Corsi Percentage thing?

Haven’t you had enough yet? Not to worry, we’ll dig deeper into the changes in shot attempts in Part 2, which (hopefully) will be up soon.

Data

All data used in this article was taken from War On Ice. Data is for 5v5 situations only, and covers all games from the 2005-2006 season to the end of the 2013-2014 season, excluding playoffs.

Tagged with: , , , ,
Posted in Score Effects

Predicting Free Agent Salaries

With the NHL draft behind us, attention across the hockey world has turned to July 1st, when unrestricted free agents will be free to sign with the team that will give them the best shot at winning a cup the highest bidder, and fans of all franchises pray that their GM doesn’t screw anything up for a superstar to vault them into contender status. And while every deal signed will be scrutinized from a million different angles to determine whether or not a team paid fair money, most of this discussion will end up conflating the ideas of value (will Martin St. Louis still be putting up 20+ goals at age 42?) and price (should the Bruins, err Flames, really pay Dougie Hamilton $5.5MM per year?).

This, unfortunately, is a rather large mistake, because what teams pay for a player and what that player is actually worth to a team are two critical yet extremely different questions. Worth (or value) is what a player adds to a team on the ice (in goals or wins), and must be measured based on a player’s total contribution (see, for example, WAR on Ice’s Wins Above Replacement, Hockey Reference’s Point Shares, or Hockey Prospectus’ Goals Versus Threshold). Price, on the other hand, is simply what a team is willing to pay for a player, in contract dollars or cap hit. While ideally these numbers would align, the reality is that teams often value observed historical results more than they should. Teams tend to pay for basic counting stats while ignoring other potentially useful indicators of future success (such as general shot generation or prevention) or contextual factors (such as quality of teammates).

This market inefficiency presents an opportunity for teams, as GMs that can identify which players are over or underpaid relative to their actual contribution should have a long-term advantage in a cap-restricted world. The key to taking advantage of these opportunities, however, is being able to predict what your opponents are going to do. If a general manager knows what the rest of the league will pay a player, this information can be used to help identify potential targets in free agency before the negotiating period begins. Given that marquee free agents can often sign within hours of hitting the open market, this knowledge can help teams avoid chasing after players they know will be out of their budget. On the other hand, it can also help GMs know when they can play hard ball with their own players and refuse to submit to offers that are above what the market is likely to pay. All of which is to say that there’s a lot of value in being able to guess what the other 29 clubs in the league will be willing to pay a free agent, which brings us to the main question we’ll look at in this article: can we build a model to predict how much a free-agent will end up signing for, based on his historical stats?

Read more ›

Tagged with: ,
Posted in Free Agency, Predictions

Estimating the Monetary Value of a Draft Pick

Calculating the value of a draft pick is not an easy task – there’s a great amount of uncertainty involved in any given pick, and the number of “misses” are significantly more than the number of “hits”, particularly in later rounds. Compounding the issue is the lack of a clear definition for success – while most methods to date have defined a successful pick as a player who reached the 200 NHL game mark, this approach is flawed in that it ignores differences in contribution and is biased against higher round picks. In this article, we’ll look at how to address these issues with a few simple changes to the standard approach, and use a novel evaluation approach to build a model that should more accurately estimate the value of an NHL draft pick.

Read more ›

Tagged with: ,
Posted in Draft

Puck++ Playoff Prediction Challenge – Round 2 Results

With round 2 of the playoffs wrapped up and a few days of rest penciled into the schedule, there’s no better time to check in on how the entrants in the Puck++ Playoff Prediction Challenge are doing. As a reminder on the format, each entrant provided the probability that each team would win a given series, with each entry scored using the Brier Score. Entrants who provided picks for round 1 but missed the round 2 deadline were given a default guess of 0.5 in each series to keep things interesting and to allow people to jump back in for round 3. I’ve also included a Naïve set of predictions, which were generated using the regular season results as a measure of a team’s “true talent”.

With all that said, let’s take a look at who’s sitting in the driver’s seat heading into the conference finals:

Read more ›

Posted in Predictions

Puck++ Playoff Prediction Challenge – Round 1 Results

With round 1 wrapped up and the division finals now underway, it’s time to take a look at how the entrants in the first ever Puck++ Playoff Prediction Challenge performed. The table below has each entrants total score (using the Brier Score, lower is better) as well as the number of series that each entrant correctly predicted.

Read more ›

Posted in Playoffs

2015 Playoff Predictions

After three seemingly never ending nights without hockey, the playoffs are finally upon us, with the Habs and Sens, and Caps and Isles set to kick off the postseason tonight at 7. As in years past I’m going to be posting predictions for each matchup as well as overall cup odds throughout the playoffs. Unlike the last 2 years, however, I’m using a new (and hopefully improved) model to predict how well each team will do.

This new model is based partly on work that I’ve completed and written about in the past (for example, a major element is Score Adjusted Weighted Shots), and partly on work that I’ve done but haven’t written about. I’ll hopefully get into more detail on the model later, but at a high level the key factors are:

  • 5v5 Score Adjusted Weighted Shots For Per 60 (over the last 25 games)
  • 5v5 Score Adjusted Weighted Shots Against Per 60 (over the last 25 games)
  • Penalties Drawn Per 60 (full season)
  • Penalties Taken Per 60 (full season)
  • Powerplay Weighted Shot Differential Per 60 (full season)
  • Penalty Kill Weighted Shot Differential Per 60 (full season)

Both the Powerplay and Penalty Kill shot weights are different from the even strength weights, but the principles used in deriving the weights are the same. All these factors are scaled against league average and then used to determine each team’s advantage on offense, defense and special teams. Based on these advantages we can predict each team’s likelihood of defeating their opponent in a single game, and extrapolate these single game numbers to find a total series probability.

So who does our new model think will be accepting the Cup from Gary Bettman this spring?

Read more ›

Tagged with: ,
Posted in Predictions

Puck++ Playoff Prediction Challenge

Do you like the playoffs? Do you like predicting things? And do you think the NHL’s claim that SAP has a guy who can get 85% of playoff series right is a bit unbelievable? Then do we here at Puck++ have a contest for you!

For the first (and hopefully not last) time, Puck++ is hosting a Playoff Prediction Challenge. To enter, all you need to do is provide the probability of the home team winning each series, round by round. The entry form and a full set of rules are available here – entries will be accepted at any time, however you’ll only be able to provide predictions for any series that has not yet begun, so don’t wait until the last minute to get your picks in.

What do the winners get? Glory, and the right to call yourself the smartest person in the hockey world for the next year*.

So what are you waiting for? Entering takes as little or as much time as you’d like, so fire up R or Python or your crystal ball and get predicting!

*Puck++ provides no guarantee that you will be recognized as the smartest person in the hockey world.

Tagged with: ,
Posted in Playoffs
Follow

Get every new post delivered to your Inbox.