Last week we looked at data from the NFL Scouting Combine and used MicroStrategy and R to examine the strength of Combine performance as a predictor of future NFL success and draft selection quintile. This week we’ll look at how our models fared in the real world (spoiler alert—not great!), and look to make some predictions about the year to come.
Now that the 2018 NFL Draft is officially in the books—all 3 days, 256 picks, and 14 and a half hours of it—we now know where (most of) the 2018 rookie class will start their NFL careers, and we can revisit our predictions to see how we did. Given that we built a model last week to predict expected draft quintile, that seems like the logical place to start.
So how did we do? Not great—our true positive rate, which is the percentage of the draft quintiles that we predicted correctly, was only 29%. And it turns out that the model is really bad at predicting where quarterbacks will be drafted. How bad? To the tune of predicting Baker Mayfield, Josh Rosen, and Sam Darnold going undrafted.
Why is the model so bad at predicting quarterbacks, you ask? We have a theory about that! Most quarterbacks don't really fit the mold of the typical NFL player and their Combine numbers are comparatively poor when compared with other players drafted in the early rounds. If you're only looking at raw Combine results, Sam Darnold (who is forecasted to be a Pro Bowl-caliber QB) ran the 40-yard dash slower than the average linebacker, and had a vertical leap that was worse than some linemen! No wonder they're hard to forecast.
But all is not lost! Data science and model building are, by their very nature, iterative processes. And in building this model, we learned some valuable things that we can use to make the next iteration of the model even better.
- Divide into positional groups: While height and weight are decent proxies for position, they're not perfect. For instance, it's pretty safe to assume that someone who is 6'3, 288 lbs is a lineman, unless that person is Jared Lorenzen, in which case he's just a very large quarterback, whose success would depend on completely different skills than a defensive end. Putting players in positional groups (linemen, skill positions, defensive backs, etc.) would help us refine the model.
- Account for college conferences: Not all conferences are created equal. The SEC has a definite edge over the Colonial Athletic Association (CAA), and the model should in some way account for that. There's a reason someone's playing at Alabama rather than my beloved W&M, and this should be reflected in the model. One idea would be to add a binary metric for player in a Power 5 conference, another would be to weight programs according to their performance in the year that a player is drafted.
- Look at college statistics: While college stats can be tricky to compare (due in large part to huge disparities in competition), they do provide concrete performance data and should be accounted for in some way in a model attempting to predict how someone will perform in the NFL.
One thing that did jump out to us during our research is that historically, draft quintile is actually a pretty good predictor of rookie success, as measured by AV.
Now that we know the results of this year's draft, we can apply our model to the 2018 class and make some (bold) predictions. Here's our hot take on the 2018 draft and which rookies to keep an eye on this season:
Top 10 Projected Performers by All Purpose Yards
- Nick Chubb, 5'11", 227 lbs., RB, Round 2—Pick 3, Cleveland Browns, 637 yards
- D.J. Moore, 6'0", 210 lbs., WR, Round 1—Pick 24, Carolina Panthers, 581 yards
- Anthony Miller, 5'11", 201 lbs., WR, Round 2—Pick 19, Cleveland Browns, 578 yards
- Chase Edmonds, 5'9", 205 lbs., RB, Round 4—Pick 34, Arizona Cardinals, 565 yards
- Bo Scarbrough, 6'1", 228 lbs., RB, Round 7—Pick 18, Dallas Cowboys, 560 yards
- John Kelly, 5'10", 216 lbs., RB, Round 6—Pick 2, Los Angeles Rams, 540 yards
- Daurice Fountain, 6'1", 210 lbs., WR, Round 5—Pick 22, Indianapolis Colts, 523 yards
- Kerryon Johnson, 6'0", 213 lbs., RB, Round 2—Pick 11, Detroit Lions, 517 yards
- Nyheim Hines, 5'8", 198 lbs., RB, Round 4—Pick 4, Indianapolis Colts, 517 yards
- Mark Walton, 5'10", 202 lbs., RB, Round 4—Pick 12, Cincinnati Bengals, 497 yards
Some food for thought, but you probably shouldn't bet your whole fantasy season on our projections.
All this reading about the Combine and number crunching got us thinking—how would we do at the Combine? No promises, but the wheels are in motion for the first annual MicroStrategy Marketing Combine™. Stay tuned—depending on how embarrassing the results are, we might even publish them! Be sure to check back at the end of the NFL season when we'll take a look at how our projections actually fared.
Interested in learning more about using MicroStrategy with R? Check out the MicroStrategy R Integration Pack on GitHub and download MicroStrategy Desktop today to get started.