There are two objectives at FantasyRef.ca
Objective #1: Determine which forecaster had the best projections
Objective #2: Determine the accuracy of point total projections
While these two objectives may seem to be the same, they are significantly different. Certainly the best fantasy projections are going to have accurate point total projections, but other factors need to be considered to determine which forecaster has the best overall projections.
Consideration #1) The accuracy of the projected rank of players is more important than the accuracy of their actual point totals.
Decisions to draft players are determined by the projected order that the player is going to finish in. Whether it be goals, assists, or points, the relative order that the player is projected to finish is the most important. Ultimately, as long as a player finishes in the rank that he was projected to be in it doesn’t matter if a player gets 110 points or 93 points. If the player was projected to finish first in the league for a stat and did, you do not lose any points in your fantasy league. However if the player actually obtained his projected point total stat. but finished with lower rank, then you would be losing points in your fantasy league. For example, if you picked the number one overall ranked player for points and that player actually scored the exact projected points that were forecasted but finished 2nd overall in total points, you would not have gained the most potential points in your fantasy league as you could have. Therefore the accuracy of the projected rank is more important than the accuracy of the players’ actual point totals.
We have based our rankings purely on the accuracy of the projected player ranks. The accuracy of projected point total have also been calculated, but for purely for the sake of analysis.
Consideration #2) All projections of the same type (Pts, G, A, etc) don’t not have the same merit.
When determining which overall set of projections are the best, consideration of the importance of the projection is required. It is more important that fantasy projections are accurate for the players that will give the most points overall. Thus as point totals decrease, so does the importance of the accuracy of the projections. Therefore, more value needs to be given to the projections of players that will get the most points.
We have created a points system that factors in the difference in importance by weighting the points for each player range based on actual point total of players in that range. See Our Calculations for more details.
Consideration #3) Every stat category has to be treated equally.
Not all fantasy leagues are set up the same way. For example, some leagues put emphasis on goals and count them as double compared to assists, others value goals and assists equally. Power play goals can be worth 1.5 points, and game winning goals worth 2. The possibilities are endless. It is impossible to create comparison and rankings for all scenarios.
When determining the overall rankings each stat category, we assigned equal value to each statistical category. See Our Calculations for more details.
Consideration #4) Offensive evaluation was limited to the top 300 and ties, Defensive evaluation was limited to the top 40 and ties.
Depending on the forecaster, a certain number players were included in their projections. This number varied depending on the forecaster. Many have over 300 but some have only 300. In order to fairly compare and rank the projections of forecasters, a common number player projections was needed to be used. For defensive players, most of the projections only included 40 players.
Therefore, we used the maximum number of projected player stats that all forecaster had in common: 300 and ties (offensive players) 40 and ties (defensive players).
Consideration #5) Goals, Assists, Points, Wins and Shutouts were used for evaluation.
Some fantasy projectors went as far as to include TOI and the number of shots taken in their fantasy projections. Not all did.
We used three statistical categories for offensive rankings (Goals, Assists, and Total Points) and two for defensive rankings (Win, Shutouts) which were common to all forecasters’ projections.
Consideration #6) An adjusted rank order was used for each stat.
Many fantasy projectors included a rank beside a player which ran 1 through the number of players in their projection. Sometimes the forecasters would give players with the same totals a different ranking. For example two players with a point total of 45 could have the ranking 100, and 101. The way that this was determined varied. Frequently it was due to alphabetical order, and in some cases it was due to the number of goals the player scored. As mentioned in consideration #3, each statistical category should have the same merit. This also should be for projections of players that have the same forecasted point totals.
Ee created an adjusted rank for all players which reflects the equal merit of players with the same projected point totals. Thus, in the previous example, the two players rank would be adjusted to be the same for that stat (Each player would have a rank of 100). See Our Calculations for more details.
Remember, it is up to the viewer to determine how the rankings are interpreted for their pool scoring system.