by PuckStopsHere on 11/22/08 at 01:57 PM ET
There has been a series of posts on the blogosphere about the value of the +/- rating stat. Since I have a longstanding interest in sabermetrics and hockey I thought I would comment on it. The series of posts begin with David Staples of the Edmonton Journal writing on his Cult of Hockey blog. He wrote a post called Why Plus/Minus is a Rotten, Useless, Misleading and Irrelevant Stat for NHL Players. If we go beyond the hyperbole of the title, he makes the point that +/- ratings can be misleading because (like any other statistic) there is a context to the numbers which is not clear from one number alone.
As an example, he writes:
For instance, this year Marian Hossa of the powerful Red Wings is plus-8. But last season, playing with the hapless Thrashers most of the year, Hossa was minus-14.
Hossa is pretty much the same strong player this year as he was last year, but his plus/minus this year would make you think he’s utterly outstanding, while his plus/minus last year would make you think he was a mediocre bum.
The plus/minus stat tells us a lot about the strength of the team that a player is on, but very little about the strength of an individual player. It’s a team stat, not an individual stat, yet it’s used constantly by fans, reporters and commentators as an individual stat.
This is a bit of a strawman argument in that nobody with any hockey sense would ever argue that Marian Hossa was a mediocre bum last season. It is quite clear that the main difference in his +/- rating from one year to the next is the change in teams. That in no way makes +/- useless. It merely means that the context of these numbers is important.
Responding to that post, James Mirtle responds that +/- can be junked as a stat because there are better metrics to judge the value of individual players. He is correct that better metrics exist. He is correct to point to the work of Gabriel Desjardins at behind the net. Desjardins treats */- as a rate stat by using +/- per 60 minutes of 5 on 5 play. He then compares a given player’s plus minus rate when he is on the ice to that of his teammates in order to get a roughly team independent number. He then goes further to establish a context for those numbers by calculating the time averaged quality of teammates and quality of opposition. He cannot give one number to rank NHL players (which is quite likely impossible in a complex continuous sport - as opposed to one where plays are discrete events as in baseball) but he makes a significant step in that direction and better allows us to understand what is going on in the hockey games.
Mirtle argues that +/- being readily published and used in analysis can lead to some ridiculous events. For example he sees the 2006/07 Selke Trophy votes that Thomas Vanek of the Buffalo Sabres received as an example of this. Vanek had a +47 +/- rating and led the league. However, he did not play a defensive role on his team. Largely, Vanek was an offensive player who played on Buffalo’s second offensive line. Usually, Daniel Briere and Chris Drury were on the number one line that drew the opposition’s best checkers. Vanek played very well in his role playing against weaker opposition.
The problem is a fundamental misunderstanding of what +/- captures. +/- ratings are the difference between the goals scored and goals allowed when a player is on the ice (largely in even strength situations). Because they are the only of the well reported hockey stats (GP, G, A, Pts, PIMs) that has any attempt to capture defensive play. However, it does not directly capture defensive play at all. A top scorer on a good team who plays merely passable defence will usually get a good +/- rating. That said it should be possible to isolate players who play well defensively from looking at +/- ratings in the right context.
There is far too much information available in the behind the net +/- analysis to be published in a simple boxscore or in the hockey statistics usually listed in a newspaper. I think the best single number summary available might be +/- ratings. It conveys some information. It is clear that a player with the highest +/- on his team is doing something right and the player with the worst +/- on his team probably has some problems. With a little context, deeper conclusions can be drawn.
I believe that a further step to understand +/- would be in this proposal. I proposed that individual goals against and goals for averages could be calculated for individual players. They could be compared to time averaged goals for and goals against averages for their opponents to give an offensive and a defensive +/- rating that is independent of the quality of opposition. This would be a further step in the understanding of hockey sabermetrics. It is a big job to calculate. In my spare time I made some progress before mothballing the project as the new hockey season got underway. I hope to find the time to return to it someday.
The behind the net analysis can be criticized. Since players are compared to their teammates (their team’s performance when they are off the ice) this can lead to problems. If we assume that the best team in the NHL is made up of 20 interchangeable players who are exactly the same as one another, we can see this. All of these players are the best players in the NHL. Each of these players will have behind the net +/- rankings of zero. There would be no difference between their on and off ice +/- ratings (as the players when they are off the ice are exactly the same as when they are on the ice). Thus it would appear that some lesser player on a different team would have a higher rating.
The question is how often do these situations occur in the NHL today? In a parity filled league the approximation that each team is not different enough from the others to cause considerable problems would be made. That isn’t too poor an approximation in today’s NHL. Nevertheless, David Johnson of Hockey Analysis is a critic. He criticizes the behind the net method, his main example of an argument against it is wrong. He argues that Kris Draper is incorrectly seen as a poor player last season in this method. The problem is that Kris Draper did not contribute much to the Red Wings.
In Detroit, the top line with Pavel Datsyuk and Henrik Zetterberg was very good defensively and usually played against the top offensive unit on their opposing team and scored well in that situation. That is what made the Wings such a successful team. Kris Draper played on a more traditional checking unit and was almost never on the ice alongside Datsyuk or Zetterberg. Since the top offensive unit played against Datsyuk and Zetterberg, Draper played against lesser opposition. Despite that he rarely scored (he has 17 points last year). He was not able to keep the weaker opposition from scoring enough to be too valuable to his team. While there are cases where the quality of teammates influence the on/off ice +/- ratings, they are not as big as this case that David Johnson incorrectly cites.
On the Kukla’s Korner website, we see the successes of the behind the net system. On The Forecheck listed the top MVP candidates based on a system derived from the behind the net system. He lists Alexander Ovechkin as the current MVP this season. That is a bit of a surprise considering Ovechkin is currently 11th in the scoring race. I would pick his teammate Alexander Semin as the MVP at this point. This is largely because he has more points and thus has been more clearly involved in the goals that he has scored - as he was credited with a goal or an assist more often than Ovechkin. It is quite possible that the difference between them in their behind the net ranking is largely due to random fluctuations and nothing meaningful at this point. It is necessary to note that Washington is likely a team that would lead to its stars having high behind the next rankings. They have a lack of depth. As such, the stars will be better than their teammates by a larger margin than players on deeper teams such as San Jose or Detroit.
Forechecker goes on to list the worst players so far this season by his metric. This group contains a large number of checkers. Players such as Zdeno Chara, Jay Bouwmeester, John Madden, Sami Pahlsson and PJ Axelsson, who are consistently lined up against the best offenses in the NHL. These player’s jobs are to hold the scoring rate of these stars down and while doing that have not scored much themselves. It is the tough quality of opposition that keeps their ratings down. I think the truly worst players in the NHL are known to their teams and protected from tough assignments. Thus they would have a hard time getting ratings as poor as these players on The Forechecker list. I submit that Brett Lebda has probably been the worst player in the NHL at this point - though his lead is small.
Is +/- a misleading stat? It can be without the context. At the same time goals can be a misleading stat without the context. In 2005/06, Jonathan Cheechoo of the San Jose Sharks led the NHL in goals with 56. There is no reasonable way to argue he was the best player in the NHL that season. In fact, the MVP was won by teammate Joe Thornton who only scored 29 goals. Would anyone argue that this makes goals a useless, misleading stat? I hope not. Anyone with some hockey sense would realize that a main reason Cheechoo scored so many goals is from Thornton setting him up. Cheechoo’s league leading goal total is a testament to how well Thornton played.
+/- can be a very useful stat. It is a starting point to some successful sabermetric analysis of hockey. Likely, further developments in hockey sabermetrics will come from a +/- direction. It requires a context to make sense of the number. Unlike some of the other numbers we see (such as goals scored) that context is a little more complicated so we often have less of an intuitive feel for it, but that doesn’t make it a poor statistic. It is very valuable and I have enriched my understanding of hockey by studying it.
Be the first to comment.
Add a Comment
Please limit embedded image or media size to 575 pixels wide.
Most Recent Blog Posts
About The Puck Stops Here
Who am I? A diehard hockey fan.
Why am I blogging? I want to.
Why are you reading it? ???