Can a numbers-based approach outdraft QMJHL teams? I spent the last year trying to build an advanced predictive model to see if this was possible. As a test, I did a re-draft of the 2013 draft, since the majority of those players finished their QMJHL careers this season. Using exclusively information that was available before the draft and without taking into consideration any subjective factors (scouting, interviews, team needs, etc.), my model outdrafted every single team in the 2013 draft in terms of drafted player value (APV). While APV is not a flawless metric, it does correlate very highly with winning games and has successfully predicted the previous 10 President’s Cup winners. There is certainly some luck involved and I only tested one draft, but my results validate that there is value in looking at numbers in depth to help make draft decisions.
My predictive model was inspired by two NHL-based models. The Prospect Cohort Success (PCS) model was developed by Josh Weissbock and Cam Lawrence and, in my opinion, is the best hockey prospect evaluation model ever built. In fact, the model was so good that within a few months, both Weissbock and Lawrence were hired by the Florida Panthers, and the model was taken down from public view. The other model, called DEV, was created by Zac Urback, who now works for the Colorado Avalanche, and Hayden Speak. My learnings from their work was invaluable in helping me construct my model.
My model is based on historic data from the draft years of players from 2004 to present. Combining that data with my created metric called APV, I programmed a predictive model using a professional analytics platform. The model itself is an ensemble of regression and classification decisions trees (such as random forest variants). These algorithms all built upon themselves to optimize the output. So my “model” is actually thousands of different models working together.
To test my model, I re-drafted using a simple approach:
- Always draft the top rated player available, regardless of position
- Only draft American players in the last 2 rounds, since they’ve historically been low probability draft choices
I had expected my predictive model to perform well, but I didn’t expect such an overwhelmingly positive result. On average, my model drafted twice the expected APV given a team’s picks. I measured the expected APV of a draft pick by calculating the average APV for that pick from 2004 to 2012, and then building a logarithmic equation that best fit those numbers.
My model was especially good at drafting under-valued players. The following players are the main reasons why my model outdrafted every team:
- Daniel Sprong rated as the best player (was drafted 16th)
- Thomas Chabot rated as the 6th best player (was drafted 22nd)
- Mathieu Joseph rated as the 9th best player (was drafted 51st)
- Samuel Laberge rated as the 13th best player (was drafted 95th)
- Alex Barré-Boulet rated as the 19th best player (was drafted 102nd)
- Étienne Montpetit rated as the 40th best player (was drafted 137th)
- Tyler Boland was rated as the 106th best player (was drafted 180th)
My model certainly had its fair share of players who didn’t pan out (such as Alexandre Jacob and Marc-André Gauvreau), but still proved very effective at finding high potential players given their underlying numbers.
Player Career Projections
My predictive model produces more than simply a player’s projected APV. Here’s an example of all the information that is produced on a player:
- Predicted career APV: 1336
- His actual career APV was 1280.
- Volatility Index: 55
- This measures the statistical confidence in the projection, the lower the score, the “safer” the projection. Score of 55 is a slightly above average risk factor.
- Predicted APV per season:
- Season 1: 243
- Season 2: 336
- Season 3: 378
- Season 4: 379
- Projected most probable player role per season:
- Season 1: Top 6 forward (55%), bottom 6 forward (45%)
- Season 2: Top 6 forward (91%), bottom 6 forward (9%)
- Season 3: Top 6 forward (71%), not playing in QMJHL (17%)
- Season 4: Top 6 forward (69%), not playing in QMJHL (27%)
- The model correctly predicted that Sprong would spend parts of his 3rd and 4th seasons in the NHL.
- Projected comparable players:
- 76% similarity with Benjamin Breault
- 64% similarity with Brandon Hynes
- 60% similarity with Chris Doyle
- 57% similarity with Guillaume Asselin
- 55% similarity with Brad Marchand
Overall, this assessment that is based exclusively on quantifiable data ended up being quite accurate of how Sprong’s career in the Q panned out.
Team by Team Re-drafts