The 2013 Season’s Co-Most Fascinating Team: The New Jersey Devils

The lock-out shortened season was chock full of surprises and absurdities, including the Toronto Maple Leafs’ first playoff appearance in nearly a decade. They rode into the playoffs on the massive PDO wave driven by an allegedly unsustainably high shooting percentage and excellent goaltending. But that was supposed to be a fluke; after all, the test cases for regression to the mean in PDO were consistent with the idea that you couldn’t out-“luck” your problems consistently.

And yet here we are, over a month into the full 2013 season and the Leafs are once again riding that big wave of team shooting and save percentages right to the top(-ish) of the East. Most analysts are predicting heavy regression to the mean for the Leafs and a rude awakening for head coach Randy Carlyle & Co.’s conception of “quality possession”.

Sean McIndoe (Down Goes Brown) had an excellent piece in Grantland about what this season’s Leafs performance means for the hockey analytics community. Basically, the idea is that their money is where their mouth is.

But for all this discussion of the statistically impaired Maple Leafs being the poster boy of small sample sized success on the back of unsustainable performance, there is surprisingly little said about what could be another team that’s challenging analytics.

While the Leafs were busy allegedly playing unsustainably well during the lockout season,  the New Jersey Devils who were fresh off a Stanley Cup final appearance were doing just the opposite. From reading the stuff available about the Leafs, you’d think regression to the mean only went one way when it’s actually symmetric (meaning that both teams below and above are affected by it). The Devils have strong possession numbers, some of the best in the league in fact and when paired with a brutal PDO it indicates that the fundamentals are there but the Devils simply are getting unlucky (or horrific goaltending performance).

During the 48 game 2012-13 season the Devils had the best 5v5 FF% in the league with 55.7%, just edging out their recently crowned rival the Kings. But the Devils PDO was 976, a very low number and something analysts at the time pointed out would regress upwards to the mean. The Devils sported the 25th best 5v5 sv% in the league at 0.912 and the 28th best sh% at 6.42%.

So the Devils bought out Hedberg and sent their 9th overall pick to Vancouver for Corey Schneider. Surely, with this upgrade between the pipes and the acquisition of forwards Jaromir Jagr, Michael Ryder and Ryane Clowe this new-look New Jersey team would somewhat offset the loses of David Clarkson, Zach Parise and Ilya Kovalchuck; experience some puck luck; and see that sh% and sv% rise on the wave of upwards regression to the mean. But the Devils are 14 games into this post-lockout season and sit 7th in the Metropolitan division. It’s still early and the race in the Metropolitan division is pretty tight (outside of first place) but early indications are not good for the Devils.

The Devils are currently 7th in 5v5 Fenwick For % (which measures the percentage of unblocked shots taken by that team) at 52.3% but sit 30th in the league with a PDO of 957. New Jersey has the third worst 5v5 sv% in the league  and sit 26th in the league with a 5v5 sh% of 5.8%.

That’s a lot of numbers, so what exactly is the point here?

The point is not to stomp up and down that hockey analytics are wrong, that the Leafs have figured out a way to consistently snipe goals and that the Devils are a team full of plumbers that can throw the puck on net.

The point is partially that we need to give this season time to play out and give unsustainable patterns are chance to correct themselves.

But the big point is really that if traditionalist and analytics people alike want to cast this season as the proving grounds for some kind of dichotomous dick-measuring competition, they had better look at the allegedly unsustainable play of the New Jersey Devils as well. The Devils, who don’t get much spotlight, deserve to be just as much to be this season’s co-most fascinating team with the Maple Leafs as test cases for the power of hockey analytics.

Advertisements

Defense. Defense. Defense.

Since my last post, the Toronto Maple Leafs have gone 3-3-0; they have scored 17 goals, given 14, and have accumulated an additional six points to tighten their stranglehold of the top of the Eastern Conference.  Even though the Leafs are winning games and are atop of the conference, the most impressive detail has been the goaltending – the Leafs have given up 30 goals in 13 games, which brings them to a surprising 2.31 GAA – good for 6th overall in the NHL.

One thing I really enjoy about the hockey blogosphere is that I get to read Leafs-centric posts like Anthony Petrielli’s Monday Notebooks that explores the past week of hockey that the Leafs have played.  Petrielli pours a lot of his own time into studying the nuances of hockey and applying that deep well of knowledge into his weekly breakdown of the Leafs.  Obviously, his work is a tremendous read, often pointing out seemingly minor details that have had a huge impact on game.

Petrielli’s methodology is kind of what I would like to do here with this blog.  I don’t consider myself to have the same understanding of hockey that he does, but I do think I can try to explore some of the other nuances of the game – such as analytics.  The thing is, analytics to some readers is considered a dirty word; a clinical perversion of the chaotic state of hockey.  This kind of mindset is a little unfair – the state of hockey analytics may not be very strong just yet, but it is getting there.

Today, I am going to talk about the relationship between shots and score effects and post some stuff I have been monitoring.

James Reimer & Jonathan Bernier

Note: This section is a bit disjointed as I continued to write, so if you need clarification, post a comment and I will try my best to unravel the mess that are my thoughts at the moment.

One thing I have been keeping a close eye on is to see which goaltender seems to positively influence possession.  When the Leafs acquired Bernier, I posited that the Leafs wanted a goaltender who could handle the puck.  If it is true that the Leafs wanted a puck-handler who could shut down the forecheck quickly, then they seem to have picked a pretty decent goaltender to trade for.

With that said, Bernier has a Corsi For percentage of 45.4% to Reimer’s 40.2% in 5 on 5 close situations of two goals or less.  If you remove blocked shots, the Fenwick For percentage favours Bernier 44% to 38.2% in the same even strength situations.  It has been said that goaltenders cannot influence possession in a statistically relevant way, but I’m not too sure that is the case.  Last season, Ben Scrivens and Reimer had a CF% of 44.6% and 42.8% respectively.  Perhaps it is my own takeaway, but it seems to show that Reimer does seem to negatively impact the Leafs possession woes.

The more I think about it, the team defense hasn’t caught up to the development of the Leafs goaltending; basically, the team defense is light-years behind the stellar goaltending we are watching from Reimer and Bernier.  So as a whole, the Leafs’ inconsistent play is reflective the non-linear team development.  If that is the case, once the defense starts to gel, we should start seeing better shot suppression.

We’re only 61 games into the reverse Corsi that the Leafs bring to the ice, but it’s just enough to get a glimpse of the team’s seeming inability to hold onto the puck long enough to put a dent into someone’s spreadsheet.  Someone recently mentioned that the Leafs scored the most first goals in the NHL last season – which turned out to be true to a tune of 32 games with the Leafs having scored first.

When you apply the score effect of two goals lead or less, the game slowly begins to open up.  If you think about it, the application of score effects are important because we can see that Scrivens had a -5 goal differential last season compared to +12 from Reimer – the quality of goaltending is important here.  This brings me to my next point: the Game 7 collapse against Boston highlighted the team’s inexperience of shutting down a desperate team.  The twist to having led the league in goals scored first is that the Leafs only won 19 of those games.  They simply don’t have a lot of experience with handling pressure.

This is partially why I am convinced that it’s not the system, but the team defense – the inexperience is constantly on display.  A missed assignment here, a missed assignment there – and the shots add up.  If anything, I think the system Randy Carlyle has implemented is protecting the missed assignments because it assumes that there will be a lot of mistakes made.  That somewhat explains why the defense collapses when a shot gets through to the net.

Obviously, a goal is the end result of preceding events leading up to it, so the differential not an analysis really fleshed out – but the Leafs are a quick strike team that feasts on turnovers, open-ice, and transition play.  So while goal-differential is a pretty one-dimensional analysis, it is another way of trying to figure out the perspective of management, so I think it is a pretty important detail.  The refrain from Carlyle is a pretty good way of putting it: “We just need them(Reimer and Bernier) to give us a chance to win.”

Of note: QuantHockey has the Leafs as the youngest team in the NHL.  That’s not going to change significantly with Bozak, Kulemin, and Fraser eventually making their way back to the line-up.  In addition, the Leafs defense is currently listed as the sixth youngest.

Location, Location, Location

A growing consensus among social media and analytics is that the Leafs don’t really keep shots to the outside – on average.  I think this is a misrepresentation of the system the Leafs have in place because they generally keep shots to the outside for the most part until the score effects take place (Note: This is an observation).

If you look at the shots the Leafs faced at even strength last season, they led the league in shots allowed from outside 20 feet, but were generally league average in shots allowed inside 20 feet.  I explored this awhile back because I didn’t think analytics were representing the Leafs’ possession issues in an impartial or fair way.  Obviously, 20 feet more or less is arbitrary, but it did clear up the shot location picture quite a bit.  I may go back at some point to look at analyzing shots outside 30 feet and inside 30 feet though.

One thing about score effects: they are a pretty accurate way of looking at how teams perform with a lead over long period of time.  I thought last night’s game against Edmonton highlighted how important it is to monitor score effects because they are generally reflective of how the game opens up league-wide.  Last night’s game also highlighted the shot location picture a bit.  The Leafs managed to shut down the Edmonton Oilers, but I’m not sure that they did it with authority.  This goes back to the shutdown issue I brought up with regards to the team defense. Check out the three periods.

Edmonton Toronto -- First Period

Edmonton Toronto -- Second Period

Edmonton Toronto -- Third Period

If you look at the first two periods, the Oilers tried to get shots through on Reimer’s glove side (when are teams going to learn that he actually has a pretty decent glove hand?).  While the Leafs struggled a bit with keeping shots outside last night, Reimer was up to the task.  The Oilers are a highly skilled team, so it was no surprise that they were able to generate scoring chances.  After two periods, Jonathan Willis noted that the scoring chances were even at 11 for both teams.  But the important distinction is this: the Leafs were suppressing shots pretty well.  Through one and a half periods, they gave up 12 shots – that’s 24 shots over the course of a full game.

In addition, the Leafs and Oilers were pretty much neck to neck at Fenwick For percentage throughout the game.  This is to be expected because most teams are generally around the 48-52% FF% range.  But once the Leafs scored their third goal, you kind of knew the Oilers would get desperate.  ExtraSkater has a nice highlight of the score effects after the fourth goal below.

Edmonton Toronto -- Fenwick Chart

I think the Leafs are managed in a way that they begin to settle too early once they are up a goal.  Perhaps someone with more analytical resources can take a look because once the Leafs are up a goal, they generally start to sit back more often than not.

Here are the shot locations charts from the last six games: Carolina vs. TorontoChicago vs. TorontoAnaheim vs. TorontoColumbus vs. TorontoPittsburgh vs. Toronto, Edmonton vs. Toronto

Phaneuf is a one-man show

Yesterday, rumours came up on Twitter that Dion Phaneuf was seeking a 49.8MM extension over 7-years.  This kind of rumour bugs me because we just don’t know what’s being discussed.  Damien Cox suggested that there have been zero discussion between Phaneuf’s agent and Leafs management, so take the rumours for what you will.

But that doesn’t change the fact that Phaneuf may be the most important cog to the Leafs machine.  Phil Kessel leads the offense, the goaltenders do their goaltending, but who takes on the toughest assignments while producing at better than a 0.5 point per game rate?  As I wrote last week, the Leafs are a different team without Phaneuf.  If the team doesn’t have Phaneuf, then they have no one to protect Franson, Rielly, Gardiner, and the goaltenders.

In the last six games, Phaneuf has allowed two even strength goals against the likes of Ryan Getzlaf, Jonathan Toews, Corey Perry, Patrick Kane, Marian Gaborik, Eric Staal, Alexander Semin, Marian Hossa, Jordan Eberle, Sidney Crosby, Evgeni Malkin.  

The problem is, I don’t feel that Carlyle is utilizing the first-pairing properly.  Perhaps it is his way of trying to distribute speed and skill in a balanced way so that the team is never ‘out’ of it at any point when a different pairing hits the ice, but even then, you want your best players to be your best players and you want them to eat the most minutes.

But Gunnarsson isn’t remotely close to our best player.  He hasn’t been Gunnarsson since hurting his hip.  For whatever reason, whether lack of surgery or just discomfort, Gunnarsson’s mobility has become so impacted that he’s chipping the puck out and struggling to maintain gap control.  His first pass is good when he actually makes the pass – but that has become a less frequent occurrence.

I have been a strong proponent of removing Gunnarsson – even suggesting he should be traded – and promoting Jake Gardiner to the top-pairing for the last little while now.  Gardiner brings left-side strength of speed, skating, mobility, high level of skill, a first pass, and an ability to add a left-side neutral zone threat that Gunnarsson doesn’t bring.  If you have watched Gardiner over the last several games, you have probably noticed that he’s been the team’s second best defender after Phaneuf.

If you want numbers, Phaneuf and Gardiner have played 206 minutes at even-strength together in the last three seasons: they have a CF% of 58.5%.   In close situations, they have a CF% of 49.4% in 85 minutes.  When Phaneuf plays with Gunnarsson, he’s producing a CF% of 47% — 46% in close situations.

Now obviously, the question becomes whether Gardiner can handle more minutes and tougher competition.  If the team performs better at even strength possession with Gardiner and Phaneuf on the ice together, then the Leafs should get more offensive zone starts, which means less defensive situations, which means better situational usage for the other pairings.

As we are currently seeing with Kadri, you won’t know until you take the training wheels off.  Maybe it time for Carlyle to do the same for Gardiner.  He has shown he at least deserves a shot at playing shotgun with Phaneuf.

On the battle between Bernier and Reimer and shortcomings of goaltending analtyics

So my last posting was discussing some of the issues cropping up in the field of hockey analytics.  I spoke at some length of how the rights and wrongs of analytics have essentially destroyed some of the fabric of what it means to be a fan – especially on social media where information is instantly disseminated and taken at face value.  It is great that people are taking a deeper look into the mechanisms of how hockey flows and blends itself into persistent averages, but sometimes the data looks as if the researchers are being too strict with their interpretations.  It also looks like analysts are using play-doh to mortar the gaps in the data they have finessed and passing it off as factually relevant.

Before I start, I want to make it clear I have no metadata right now.  I will be working on acquiring that via purchasing from someone or developing my own program to cull as much hockey data I can get my hands on.  Until then, my criticism will be focused on what I have and how analytics can be improved by following some of the simple rules of hockey.

A Visual Example of Contrast and Approach

Let’s start with an example of something that is trendy and topical.  Right now, the Toronto Maple Leafs have a bit of a conundrum going on – a happy one depending on who you talk to.  Quickly, do you take Jonathan Bernier or James Reimer?  As the Leafs barrel towards a tenth of their games played the debate rages on who the better goaltender is.

We know what Reimer brings – sturdy protection of the net, great size, a never quit attitude, and consistently excellent numbers that would make him the envy of most goaltenders.  But he does have his fair share of critics – notably his rebound control and a weak glove hand.  Both ‘weaknesses’ I think are overblown, but noteworthy for this exercise in excessive pedantry.

Conversely, Bernier has a slightly smaller track record.  For all of his purported talent, we just don’t have enough of a sample size to really gauge what kind of strength and weaknesses he has.   We know that he’s a bit on the smaller size, but is lean, athletic, and a bit of a hybrid goaltender in which he can play two distinct styles depending on the development of the play.

Part of the ongoing debate suggests that Bernier’s rebound control is significantly better than Remier’s, but there’s not a lot of data out there to support it – yet.  Part of the problem is that both players have small career sample size, so we don’t have a lot of numbers to really make a determination one way or the other.  Coincidentally, Eric T. from Broadstreet Hockey wrote an article on how to determine NHL goaltending performance.  I thought the .gif in the article was a really neat way to convey how important sample sizes are from an analytical and viewing perspective.

Image

In any event, both Bernier and Reimer are very different goaltenders in how they position to defend the net.  Reimer almost always goes down to take away the bottom half of the net while Bernier is a little more cerebral about his decision-making.

From my viewing of every game this season, Bernier absorbs shots exceptionally well and seems to handle and move the puck extremely well.  Reimer, in the three plus seasons I have watched him, seems to have up and down games with his rebound control; and no one would mistake him for being a puck-handling virtuoso.

The problem is, we don’t really have anything to indicate exactly how Reimer’s rebound control works compared to other goaltenders around the league – vice versa for Bernier.

In Bernier and Reimer, the Leafs have two potential elite starters, but no concrete evidence to support one player or another – at this point, the debate is between a marginally larger sample size vs. preference.

The Analytical Example of Measuring Stylistic Approaches

About a week ago, I was talking to Sasko Taskov about the goaltending battle and he brought up Reimer’s elite rebound control.  I thought it was an interesting point to make because I didn’t agree that the rebound control was elite.  Which brings me to what the point of this article is today.  A while ago, I was browsing through Twitter and found a neat article on whether goaltenders have an ability to control the number of rebounds they allow.  I thought the results were interesting, but it was the methodology that stuck out to me.

“We will define a rebound as any shot taken at 5v5 within three seconds of a previous shot, in a continuous action situation.”

I took a look at one of the links in the article that was written and researched by Gabriel Desjardins; it is intriguing how valuable rebounds are in the initial two seconds.  Basically, within the first two seconds of a rebound, you can expect just less than one in every two rebounds created to go in – beyond the elapsed two seconds of the initial shot, the ~46.5% goal opportunity has a declining effect until it normalizes around the five second mark.  This makes a ton of sense because the oft-repeated buzz-saying is to crash the net – crash the net at the time a shot is sent in net’s direction and there’s a chance the player will have an opportunity to put the puck in on the first or second touch.

Pattapiece takes a part of the idea from Desjardins’ article to focus on how often a rebound is given up by a goaltender within a time frame of three seconds.  In Pattapiece’s research, Reimer is one of the strongest controllers of shots in the NHL.  This is particularly interesting because of the company he is in – Rinne, Thomas, Quick, Rask, Luongo, Miller, Smith.  A lot of those goaltenders listed are well known for their quality work in net, so it is easy to assume that the methodology is pretty accurate.

But this methodology isn’t without its flaws.

The Definition Problem

The problem is, the language of statistics doesn’t always marry well with subjective jargon — especially jargon that kind of mixes in a bunch of random variables.

We know that rebounds are generally pretty consistent in terms of how they are defined from a viewing perspective, but that’s not logistically feasible — we tend to treat rebounds as visually dangerous ones.  In addition, three seconds is an eternity when there’s a goal-mouth scramble.  Shots taken off rebounds are generally one-touch shots as soon as the puck touches the ice.

The main problem is the league’s general conflation of shots with a rebound – it is inconsistently tracked.  So we’re basically left having to either watch the game and visually track rebounds or time consecutive shots within a certain amount of time.

Back in 2004, Alan Ryder made one of the first attempts to narrow the definition of a rebound based on the time elapsed between the initial shot and the second shot.

“I was able to identify 1,899 rebound shots by defining a rebound as a goal or shot within two seconds of another shot with no intervening “event”.

You can see the similarities between Ryder’s effort to define the rebound and Pattapiece’s.  The one second divide is pretty significant as we can see in Ryder’s initial assessment that he calculated rebounds to make up roughly 4% of total shots faced, whereas Pattapiece calculates 3.8%.  A third source from hockeyanalytics also confirms that rebounds generally make up roughly 4% of total confirmed shots.

I took a quick look at the even-strength level to see how wide that difference would be – the 0.2% difference between the two data sets is roughly 115 shots.  Where the data between three sources align together, they are still defining a rebound as being dangerous because of the elapsed time between two shots rather than none at all.

None of the works listed above tracked a rebound intentionally turned away to keep the play flowing – turning the puck away is a well-known strategy used by some goaltenders to help their defensemen manage their defensive posturing rather than absorbing a defensive zone face-off.  This is another ‘rebound’ control needed to be analyzed (or at least acknowledged).

And what of those goaltenders who snag the puck and pass it forward to a teammate?  This may make up a very minuscule and statistically inert portion of the play, but of course, that’s another example of a goaltender using his rebound control to manage the puck.

Point is, a rebound can be a lot of things, but a three second window is probably a little too broad and simultaneously narrow.  One major problem of secondly increments is that it doesn’t really mitigate the system issue – some teams prefer to accept shot and clear the rebounds, others look to suppress it quickly.

The Future of Rebound Analytics and some Suggestions

I propose that the window is narrowed to a one and two second increments – going beyond two second increments is likely a product of a failed clear by the defensive unit or a simple reset of the offensive net by the attacking team.

Secondly, one shot can produce a rebound, but doesn’t always produce a secondary shot.  Some goaltenders intentionally use the initial shot to move the puck in a harmless direction.  This particular form of an inconsequential shot needs to be statistically controlled as well.

Thirdly, there needs to be a more robust tracking feature on the NHL website or a mandate to further define what a rebound is from the league executives and players.

Overall, there are still too many holes at the statistical level to really take the works of Pattapiece and Ryder’s at face-value.  We know that a save percentage doesn’t really normalize until five seconds have elapsed, but none of the works, including Desjardins’, have really gone beyond 2005-09 and 2010-11.  In addition, none of the works are cumulative to make an accurate statistical determination at the team and individual level.

Taking my theories into a spreadsheet may show a drastic difference goaltenders with elite, good, and poor rebound control.

So this kind of summarizes an example of some of the statistical holes in analytics right now.   Perhaps someone can pick up where I left off and try to remedy the situations above.  If you got any other suggestions, give me a shout in the comments section.