So my last posting was discussing some of the issues cropping up in the field of hockey analytics. I spoke at some length of how the rights and wrongs of analytics have essentially destroyed some of the fabric of what it means to be a fan – especially on social media where information is instantly disseminated and taken at face value. It is great that people are taking a deeper look into the mechanisms of how hockey flows and blends itself into persistent averages, but sometimes the data looks as if the researchers are being too strict with their interpretations. It also looks like analysts are using play-doh to mortar the gaps in the data they have finessed and passing it off as factually relevant.
Before I start, I want to make it clear I have no metadata right now. I will be working on acquiring that via purchasing from someone or developing my own program to cull as much hockey data I can get my hands on. Until then, my criticism will be focused on what I have and how analytics can be improved by following some of the simple rules of hockey.
A Visual Example of Contrast and Approach
Let’s start with an example of something that is trendy and topical. Right now, the Toronto Maple Leafs have a bit of a conundrum going on – a happy one depending on who you talk to. Quickly, do you take Jonathan Bernier or James Reimer? As the Leafs barrel towards a tenth of their games played the debate rages on who the better goaltender is.
We know what Reimer brings – sturdy protection of the net, great size, a never quit attitude, and consistently excellent numbers that would make him the envy of most goaltenders. But he does have his fair share of critics – notably his rebound control and a weak glove hand. Both ‘weaknesses’ I think are overblown, but noteworthy for this exercise in excessive pedantry.
Conversely, Bernier has a slightly smaller track record. For all of his purported talent, we just don’t have enough of a sample size to really gauge what kind of strength and weaknesses he has. We know that he’s a bit on the smaller size, but is lean, athletic, and a bit of a hybrid goaltender in which he can play two distinct styles depending on the development of the play.
Part of the ongoing debate suggests that Bernier’s rebound control is significantly better than Remier’s, but there’s not a lot of data out there to support it – yet. Part of the problem is that both players have small career sample size, so we don’t have a lot of numbers to really make a determination one way or the other. Coincidentally, Eric T. from Broadstreet Hockey wrote an article on how to determine NHL goaltending performance. I thought the .gif in the article was a really neat way to convey how important sample sizes are from an analytical and viewing perspective.
In any event, both Bernier and Reimer are very different goaltenders in how they position to defend the net. Reimer almost always goes down to take away the bottom half of the net while Bernier is a little more cerebral about his decision-making.
From my viewing of every game this season, Bernier absorbs shots exceptionally well and seems to handle and move the puck extremely well. Reimer, in the three plus seasons I have watched him, seems to have up and down games with his rebound control; and no one would mistake him for being a puck-handling virtuoso.
The problem is, we don’t really have anything to indicate exactly how Reimer’s rebound control works compared to other goaltenders around the league – vice versa for Bernier.
In Bernier and Reimer, the Leafs have two potential elite starters, but no concrete evidence to support one player or another – at this point, the debate is between a marginally larger sample size vs. preference.
The Analytical Example of Measuring Stylistic Approaches
About a week ago, I was talking to Sasko Taskov about the goaltending battle and he brought up Reimer’s elite rebound control. I thought it was an interesting point to make because I didn’t agree that the rebound control was elite. Which brings me to what the point of this article is today. A while ago, I was browsing through Twitter and found a neat article on whether goaltenders have an ability to control the number of rebounds they allow. I thought the results were interesting, but it was the methodology that stuck out to me.
“We will define a rebound as any shot taken at 5v5 within three seconds of a previous shot, in a continuous action situation.”
I took a look at one of the links in the article that was written and researched by Gabriel Desjardins; it is intriguing how valuable rebounds are in the initial two seconds. Basically, within the first two seconds of a rebound, you can expect just less than one in every two rebounds created to go in – beyond the elapsed two seconds of the initial shot, the ~46.5% goal opportunity has a declining effect until it normalizes around the five second mark. This makes a ton of sense because the oft-repeated buzz-saying is to crash the net – crash the net at the time a shot is sent in net’s direction and there’s a chance the player will have an opportunity to put the puck in on the first or second touch.
Pattapiece takes a part of the idea from Desjardins’ article to focus on how often a rebound is given up by a goaltender within a time frame of three seconds. In Pattapiece’s research, Reimer is one of the strongest controllers of shots in the NHL. This is particularly interesting because of the company he is in – Rinne, Thomas, Quick, Rask, Luongo, Miller, Smith. A lot of those goaltenders listed are well known for their quality work in net, so it is easy to assume that the methodology is pretty accurate.
But this methodology isn’t without its flaws.
The Definition Problem
The problem is, the language of statistics doesn’t always marry well with subjective jargon — especially jargon that kind of mixes in a bunch of random variables.
We know that rebounds are generally pretty consistent in terms of how they are defined from a viewing perspective, but that’s not logistically feasible — we tend to treat rebounds as visually dangerous ones. In addition, three seconds is an eternity when there’s a goal-mouth scramble. Shots taken off rebounds are generally one-touch shots as soon as the puck touches the ice.
The main problem is the league’s general conflation of shots with a rebound – it is inconsistently tracked. So we’re basically left having to either watch the game and visually track rebounds or time consecutive shots within a certain amount of time.
Back in 2004, Alan Ryder made one of the first attempts to narrow the definition of a rebound based on the time elapsed between the initial shot and the second shot.
“I was able to identify 1,899 rebound shots by defining a rebound as a goal or shot within two seconds of another shot with no intervening “event”.
You can see the similarities between Ryder’s effort to define the rebound and Pattapiece’s. The one second divide is pretty significant as we can see in Ryder’s initial assessment that he calculated rebounds to make up roughly 4% of total shots faced, whereas Pattapiece calculates 3.8%. A third source from hockeyanalytics also confirms that rebounds generally make up roughly 4% of total confirmed shots.
I took a quick look at the even-strength level to see how wide that difference would be – the 0.2% difference between the two data sets is roughly 115 shots. Where the data between three sources align together, they are still defining a rebound as being dangerous because of the elapsed time between two shots rather than none at all.
None of the works listed above tracked a rebound intentionally turned away to keep the play flowing – turning the puck away is a well-known strategy used by some goaltenders to help their defensemen manage their defensive posturing rather than absorbing a defensive zone face-off. This is another ‘rebound’ control needed to be analyzed (or at least acknowledged).
And what of those goaltenders who snag the puck and pass it forward to a teammate? This may make up a very minuscule and statistically inert portion of the play, but of course, that’s another example of a goaltender using his rebound control to manage the puck.
Point is, a rebound can be a lot of things, but a three second window is probably a little too broad and simultaneously narrow. One major problem of secondly increments is that it doesn’t really mitigate the system issue – some teams prefer to accept shot and clear the rebounds, others look to suppress it quickly.
The Future of Rebound Analytics and some Suggestions
I propose that the window is narrowed to a one and two second increments – going beyond two second increments is likely a product of a failed clear by the defensive unit or a simple reset of the offensive net by the attacking team.
Secondly, one shot can produce a rebound, but doesn’t always produce a secondary shot. Some goaltenders intentionally use the initial shot to move the puck in a harmless direction. This particular form of an inconsequential shot needs to be statistically controlled as well.
Thirdly, there needs to be a more robust tracking feature on the NHL website or a mandate to further define what a rebound is from the league executives and players.
Overall, there are still too many holes at the statistical level to really take the works of Pattapiece and Ryder’s at face-value. We know that a save percentage doesn’t really normalize until five seconds have elapsed, but none of the works, including Desjardins’, have really gone beyond 2005-09 and 2010-11. In addition, none of the works are cumulative to make an accurate statistical determination at the team and individual level.
Taking my theories into a spreadsheet may show a drastic difference goaltenders with elite, good, and poor rebound control.
So this kind of summarizes an example of some of the statistical holes in analytics right now. Perhaps someone can pick up where I left off and try to remedy the situations above. If you got any other suggestions, give me a shout in the comments section.
The Musings of Hockey 