“Context is king. And ultra-specific predictions are the worst kind” (Sportsnet, pg.4, 2015/16 Edition).
Sportsnet’s NHL Pool Guide takes a different approach than the forecasters who we rank each year.
- They haven’t given an exact point total prediction for each statistical category, but instead have predicted a range that the player should score between.
- They have only given predictions for total points scored (not goals and assists) for offensive statistics.
- They have only given predictions for total wins (nothing for shutouts) for defensive statistics.
- They have separate rankings for each position (#1 Ranked Center, #1Ranked Defense) and not an overall rank.
- They have given predictions for only 190 offensive players.
The difference above are the reasons that we did not include them in the official evaluation of Forecasters. However, we did evaluate Sportsnet’s predictions and have compared them to the six forecaster’s predictions that were formally ranked this season.
In order to compare Sportsnet’s predictions we needed to adjust some of their predictions.
- First, we adjusted the Pts total predictions. The Adjusted Pts Prediction was determined by taking the average of Sportsnet’s high and low Pts total prediction. For example, Sidney Crosby was predicted to score a max 110 pts and a min 75 pts. This led to adjusted Pts prediction total of 92.5 pts. [(110+75)/2 = 92.5]
- Second, we adjusted Sportnet’s Pts rank for each player. Using the adjusted Sportnet’s Pts Predictions we ranked the players from greatest to least. This changed the rank of many players. For example, Erik Karlsson was ranked #1 for defence, but ranked #26 overall after using the adjusted Pts Predictions.
The adjusted Sportsnet predictions were compared to the other six forecaster’s predictions (up to 175 players, because Sportsnet only gave 190 offensive player predictions). The same calculations and ranking system were used as in the formal evaluation. Sportsnet finished 7th place overall (Figure 1).
We compared the accuracy of the adjusted Sportsnet Pts total predictions with the other forecasters (Figure 2). For their predictions of the 175 players, they were off by a total of 2264 points. An average of 12.9 pts per player. They had the second worse Pts total prediction accuracy (6th place overall), finishing well ahead of The Sports Forecasters and just behind Hockey The Magazine.
In order to compare Sportsnet’s defensive predictions we also needed to adjust their predictions. Similarly to the offensive predictions, we adjusted the Win totals by taking the median of Sportsnet’s forecasted range for each player. However, for Sportsnet’s win rank predictions we compared them in two different ways.
First, we compared them using adjusted win ranking which follows our rationale, consideration #6 (Figure 3) . When adjusting win total predictions, an overlap of specific win total predictions occurs. Thus, there is a need adjust win rank order. Sportsnet had the second best absolute error total (429) and ranked 3rd overall for W ranking. Sportsnet had the best predictions for players ranked 11 through to 15.
Figure 3. Win rank comparison with adjusted Sportsnet predictions
The second way that we compared the predictions was using Sportnet’s own ranking system. Sportsnet gave an individual ranking to each goalie and because the ambiguity of the point prediction range and the fact that defensive players (goalies) were only given one rank, it is possible to compare using this data (Figure 4). When comparing this way, there was a drastic difference in the rankings. Sportsnet again had the second best absolute error totals (439), however instead of finishing third they were ranked 1st overall with 15.52 points.
The adjusted Sportsnet W total predictions were also compared with the other forecasters’ predictions (Figure 5). Sportsnet ranked 5th for win total accuracy. They had an absolute error total of 286.5 (4th place), which leads to an average error of 7.16 wins per goalie.