What Our New Metric Can Tell Us About The Best Wide Receivers In The Draft

Projecting college receivers to the NFL is hard. For every hit (a Justin Jefferson or an early career Odell Beckham Jr.), there’s a miss (a Laquon Treadwell or an older, midcareer Odell Beckham Jr.). Still, it’s an important position to get right: Sets with three wide receivers are now the norm for NFL offenses, 1 and teams have a greater need than ever for playmakers lined up all across the field. So despite the uncertainty and the risk, starting this Thursday in Cleveland, NFL front offices will collectively roll the dice on another crop of athletic youngsters in the hope of strengthening their passing attacks.

We thought we’d join in the fun by attempting to predict which receiving prospects from the 2021 draft class will turn out to be the best. There’s evidence that the success of even the best drafting teams can be explained entirely by chance, so why not throw our hat into the ring?

Now, to be fair to the league, you can argue that the NFL does a decent job at drafting receivers. If you measure success in terms of per-game production (receiving yards and touchdowns), the two best predictors of who will thrive at the next level are players’ draft slots and the rounds in which they were drafted.

Solid evidence for an efficient draft marketplace, right? Well, maybe not. This picture of the world is confounded by the fact that production is largely driven by opportunity, and NFL teams tend to give their early round picks many more opportunities to succeed than players drafted later on. Timo Riske at Pro Football Focus has found that teams are more likely to keep a first-round pick than a later-round pick past their rookie contracts — even if the two players performed similarly. Moreover, teams control the overall passing environment into which a player steps, and they decide where a receiver is deployed based on the routes they ask the player to run. It’s not outlandish to believe that sunk costs can skew a team’s attitude toward a player and end up coloring how he’s treated, creating something of a self-fulfilling prophecy. Receivers picked higher in the draft will naturally accrue more receiving statistics, so when analysts like me try to create models to predict receiver success in the NFL, the draft will always come out looking like the most important predictor.

With all this in mind, it’s worth trying to isolate the skills that a receiver is responsible for. As USA Today’s Steven Ruiz points out, the core traits a receiver must possess to win in the NFL are the ability to separate and the ability to catch the ball. We’ve looked at which NFL receivers tend to separate more than we might expect, but we currently lack the data to conduct such an analysis at the college level. What we can estimate is a receiver’s catch rate over what we would expect.

This analysis is closely related to the one we conducted for college quarterbacks. Our model for receivers, using college data from 2011 through 2020, 2 accounts for how deep down the field each pass traveled in the air, where on the field the receiver attempted to catch the ball 3 and how catchable the targeted pass ultimately was. We also adjust for conference, team environment, the quarterback throwing the pass to the receiver and the offensive team’s head coach. All of that goes into our new metric: Catch Rate Over Expected, or CROE.

CROE is given as the share of passes a receiver caught above or below what we would expect for an average player on the same team, with the same coach and QB, in a similar on-field situation. We then take that percentage, calculate it for a player’s entire college career, and place it on a 0-100 range of all drafted college players in this time frame to give a sense for where a player’s CROE ranks historically. The highest college player in our sample was Nick Toon; the lowest was Marquez Valdes-Scantling. And it’s important to note that because CROE is trained on a group of players who were ultimately drafted by the NFL, even Valdes-Scantling’s 0 scaled CROE reflects an athlete with relatively good hands compared to the population of all college receivers. 4

The CROE leaderboard for 2021 prospects has decent face validity, topped by two highly touted Alabama receivers. The two Auburn receivers in our pool were rated among our bottom three, which matched the consensus view of both players until Anthony Schwartz ran the 40-yard dash in 4.26 seconds on his pro day and suddenly (and some would argue irrationally) shot up the mock draft boards. In any event, the leaderboard appears to pass the all-important eye test.

Both ’Bama WRs caught more than expected

Among those most often selected in mock drafts, wide receivers in the 2021 NFL draft class by Catch Rate Over Expected during their college careers

Scaled CROE places this year’s draftees on among the range of outcomes for all college wide receivers since 2011 who were ultimately drafted, on a 0-100 scale.

Source: ESPN Stats & Information Group, Grinding the Mocks

Some surprises appear near the top of the leaderboard, though. Ole Miss receiver Elijah Moore and Florida wideout Kadarius Toney are projected as Day 2 picks, but they both appear to possess a knack for coming down with the football that could be undervalued. Ja’Marr Chase is the consensus WR1 of this draft class and will likely be selected in the top 10, but LSU teammate Terrace Marshall Jr. might have the better hands. Meanwhile, Minnesota’s Rashod Bateman — a player projected as a late first-round selection — seems to have an average pair of mitts compared with other college receivers, which puts him in the lower range of drafted wide receivers (42) in CROE since 2011.

Our hubris extends only so far, however. NFL teams have access to information that outside analysts do not, including background investigations and medical checkups and histories. And perhaps most importantly, the team that drafts a player ostensibly has a plan to maximize his skills that only they are party to. 5 Given this, building a model that incorporates a player’s draft slot is an important step to making good predictions about NFL success.

To account for as much information as we could, we built an ensemble machine-learning model that stacks four separate models into one uber-model in an attempt to predict the per-game NFL production over a player’s first four seasons. 6 This larger ensemble model includes college production metrics, pre-draft athletic testing and draft slot, along with CROE. (For the 2021 draft class, we used Benjamin Robinson’s mock draft data from April 19 to predict draft slot.) With the four models together, we found that CROE was a useful feature and did add information, but it was clearly dominated by draft position. Features like a draftee’s career receiving yards, the team a receiver played for, career yards after catch, career air yards and even hand size were more important in the model than CROE was. Given this evidence, reliance on CROE as a standalone metric is questionable unless you believe that the sunk cost caveats mentioned above are compelling.

To make comparisons across years easier, and to facilitate more easily grouping players into tiers, we again scaled the model’s per-game production predictions from 0 to 100. Overall, the results track closely with expected draft position for each player, with minor variations within tiers. Those variations are interesting. Again, the consensus No. 1 WR prospect, Chase, isn’t the clear No. 1 receiver. That honor goes to Jaylen Waddle, who earns a mark of 66. For comparison, previous receivers to receive a 65 or 66 from the model include A.J. Brown, Tyler Boyd, T.Y. Hilton, Christian Kirk and Jefferson.

Who’s the most promising receiver?

Among those most often selected in mock drafts, wide receivers in the 2021 draft class by NFL projections in a model combining production metrics, pre-draft athletic testing, draft slot and Catch Rate Over Expected

Projections are on a 0-100 scale and are based on per-game production through a player’s fourth year in the NFL, since 2012. For players with fewer than four years of tenure, career per-game production is used.

Source: ESPN Stats & Information Group, Grinding the Mocks, Action Network

Meanwhile, Bateman’s lack of catches over expectation didn’t hurt his production profile at Minnesota, where he accounted for about a third of the team’s yards through the air in 2020. And he may be available to a WR-needy team like Baltimore at the end of the first round. Moore and Toney also get slight boosts, perhaps from their strong showing in CROE and the production they derived from those extra catches over expectation.

When it comes to the draft, we’re all looking through a glass darkly, and that extends even to the professionals. Thinking about the draft probabilistically isn’t just a more accurate way to view the process: It’s probably a healthier way to approach a high-pressure, high-stakes event fraught with uncertainty even in the best years. This season, with the normal draft process upended by COVID-19, NFL general managers likely feel even less confident about their big boards than normal. As a mental health prophylaxis, it’s probably better to view each pick realistically: a bet with just slightly better than 50-50 odds. 7 Spin the wheel and hope for the best. Or, if you’re like Baltimore and need a WR and have two late first-round picks, perhaps you spin the wheel twice. Dare to dream, Lamar Jackson fans.

Footnotes

1. Teams ran plays with three or more WRs on the field 62 percent of the time in 2020 — 20,471 of 32,900 regular-season offensive snaps.
2. The model was trained and tested against each charted target of every college WR since 2011, the first year for which we have data.
3. Viewed from the end zone, the football field is binned into a grid with 3.5-yard increments along the horizontal x-axis and 1-yard increments along the vertical y-axis.
4. Including all Division I and Division II athletes, for instance.
5. Though this may be a generous assumption.
6. We incorporated NFL player data from 2012 through 2020, and for players with fewer than four seasons, we used their career per-game production. Production is defined as receiving yards and touchdowns per game, converted into points. The sample was split into two buckets: 70 percent of the data was used to train the models, and the remaining 30 percent was held out to test how well the ensemble model performed at predicting NFL per-game production. The predicted per-game production from the model correlates with the out-of-sample test data at r = 0.60.
7. For first-round picks.

Josh Hermsmeyer was a football writer and analyst. @friscojosh