The Anatomy of a First-Round Quarterback Bust
After Super Bowl Sunday, there are only a few things football fans can do: cry, watch Friday Night Lights on a continuous loop or obsess over the NFL Draft. I have decided to take route three because doing number two leads to number one, and I just can’t take that any more.
In honor of the second-best event in the entire sporting world (the Daytona 500 is clearly first), we’ll be rolling out a three-part series here at numberFire about drafting quarterbacks - in real life, not fantasy - in the first round. This one will try to use college statistics to determine whether or not a player will be a bust. Later, we’ll have features on whether or not it’s worth it to take a quarterback in the first, and where this year’s crop of signal-callers stack up. I’m all tingly just thinking about it.
From 1995 to 2013, there were 47 total quarterbacks selected in the first round. For fun (actually, for this study), I sorted these passers into three categories - top, middle and bottom tier - based on their average Total Net Expected Points (NEP) during seasons in which they had at least 200 passing attempts in the NFL. In case you’re new to numberFire and need an explanation of what NEP is, you can click here.
If a player average a Total NEP score of 25 or more, I initially labeled him as a top-tier passer. A Total NEP of 0 to 25 was a middle-of-the-road guy, while anything lower was a failure. Doing this saw 18 top-tier quarterbacks, 10 middle-tier ones and 19 guys you wouldn't want leading your football team.
I should note that the categories weren’t exclusively determined by a player’s average NEP though. A player's longevity in the league and how long they have had to develop also played a role in assigning players a category. For example, Matt Leinart and Vince Young both have very average NEP scores compared to the field, but they're two players that are prime examples of what a bust looks like.
On the flip side, E.J. Manuel’s NEP last year would have landed him in the "bust" category, but I dropped him in the middle to allow for progression. I also put Christian Ponder in the middle because of his positive NEP in 2012, but I could definitely hear an argument for labeling him as a bust already.
After sorting the players into the three tiers, I collected statistics from each quarterback’s final season at their respective school to see if there was some sort of trend that could have predicted Aaron Rodgers doing Aaron Rodgers-like things or Ryan Leaf being all Ryan Leaf-y. I have listed those categories below and will go through the findings of each in the sub-headings.
The results were sobering at best for a stats nerd. However, that doesn't mean there isn't something to learn from this data. Let’s get cracking.
This turned out to be the statistic that had the largest difference between the top-tier and the bottom one. For this category, I only counted games in which the player had 10 or more pass attempts throughout their college career. The purpose was to determine whether having a larger sample size from which to evaluate made the success rate higher.
In the top tier (players with a Total NEP in the NFL of 25 or more, as discussed above), the median number of games played in college was 38. I decided to go with the median because the number of samples in each data set is so small, thus making the mean too volatile. That number dropped to 34 for the middle group and down to 31 for the lowest group. Thus, based on this data, scouts have had higher success rates in drafting quarterbacks who played more college games.
This number doesn't really tell the whole story. Two of the three lowest totals – 14 by Cam Newton and 24 by Rodgers – don’t include the number of starts from their junior college careers. That’s an additional season for each player, meaning roughly 250 more pass attempts scouts can look at to decide whether or not this person is worthy of a first-round pick.
Overall, 11 of the 47 players in the study attempted at least 10 passes in 40 or more games in their collegiate careers. Of those 11, six were in the top tier, one was in the middle and four were in the bottom tier.
At the other end of the spectrum were the 16 players that recorded 10 pass attempts in fewer than 30 games. Of those 16, four were in the top tier (including Rodgers and Newton), four were in the middle tier and the other eight were in the bottom. Half of the players with less than 30 games to their credit ended up being busts, and only 25 percent were successes. We’ll talk about what that means for a certain quarterback at Texas A&M in Part 3 of this series.
The player with the fewest number of games prior to being drafted (after you add in Newton’s junior college starts) was Mark Sanchez. He recorded 16 games at USC prior to leaving for the NFL, and we all know how that one went. Once again, Pete Carroll is a genius.
Passer Efficiency Rating
This category provided far fewer answers than games played did. The numbers were essentially all over the place in each of the three tiers.
For this stat, I only used the Passer Efficiency (not to be confused with the NEP numbers we use at numberFire) from each player in their final season at their respective school. The lone exception was Sam Bradford, who missed a majority of his final season, so I used his second-to-last year.
The correlation between a player’s Passer Efficiency and his average NEP was .1582, which means that while there was a relationship between the two statistics, the data was closer to random than it was significant.
For the same reason as above, I used the median of the category as the main basis for analysis. The medians from the top tier to the bottom were 154.85, 156.3 and 156.6 respectively. The lowest tier had a higher median Pass Efficiency than either of the top two tiers. If that isn’t definitive that this stat can be disregarded, I don’t know what is.
But not so fast, bromigo! There are still a few takeaways from this. First, the players that truly excelled in the category usually found success in the NFL. Of the 47 players, 10 had a Passer Efficiency of 168 or higher in their final college season. None of them ended up in the bottom tier. Six of those players were in the top tier with four in the middle tier.
The conclusion we can draw from that is that if you have an outstanding Passer Efficiency, you may be a safer bet at the pro level. The same cannot be said for having a low Passer Efficiency. There were eight players whose final Passer Efficiency was lower than 130. Although four of those players were in the lowest tier, three were in the top tier. This includes Matt Ryan, who has one of the highest average Total NEP per season scores of any player drafted in the first round since 1995.
The moral of the story with this stat is to use it with caution. The truly exceptional players will most likely succeed, but you should hesitate to disregard a player simply because of a poor Passer Efficiency in college.
Adjusted Yards Per Attempt
This is a stat that College Football Reference uses to evaluate quarterbacks. The formula is (Yards + (20*TD) – (45*INT))/Attempts. Note that "adjusted" in this case does not refer to strength of schedule, but rather adjusted for turnovers and touchdowns. As with Passer Efficiency, I used the total from the player’s last season in college.
In this category, the top tier certainly had an advantage over the other two, but not a convincing one. The median values from top to bottom were 9.20, 8.65 and 8.70. The correlation between a player’s AY/A and his average NEP was .1044. Again, this isn't statistically significant, but it does exhibit a relationship between the two.
The top two AY/A belonged to Cam Newton and Robert Griffin III, both of whom had Total NEP scores higher than 100 in their rookie seasons. That makes it look as if this stat has some clout. But then you look at the next five players, and that clout goes away quickly. Behind Newton in third place was Bradford, who I put in the second tier but very easily could have gone in the third tier. Then the next four players were Akili Smith, Alex Smith, Kerry Collins and Jason Campbell. Akili had an NEP of -96.06 in 2000, so he’s about as busty (not in that way) as you can get. Alex Smith, Kerry Collins and Jason Campbell never really lived up to expectations, so this stat seems like a bust, too.
This one was more just for fun than anything else. I based it off of what conference the school was in during the player’s final season there to see if there was any sort of trend. The results were great for the SEC and non-BCS schools, but not so much for the Big 12 and Pac 12.
Both the SEC and non-BCS schools have had 10 quarterbacks drafted in the first round since 1995. Both have seen five of those quarterbacks go on to be successes. Both have one player in the middle and four players in the bottom tier. So, obviously (heavy on the sarcasm), this means that Marshall is just as good at good at football as 'Bama.
The Big 12 and Pac 12/10 have seen a combined 18 of their players go in the first round since 1995; a whopping four of them were in tier one. They’re batting .222 and don’t even have a Nick-Punto-esque glove to go with it. No Bueno. To make matters worse, only four of their players even made it into tier two, meaning 10 players (or 56 percent) were busts.
I would spout out some fun stats about the Big Ten’s success rate, but, well, there’s not much data. That’s because the conference hasn't had a quarterback taken in the first round since Collins in 1995. This is despite the fact that the winning quarterback in five of the last 13 Super Bowls finished out his collegiate career at a Big Ten school (Russell Wilson was a third rounder, Drew Brees a second rounder and some bro named Tom Brady a sixth rounder). I admit, this entire article was just me trying to convince a team to draft Kain Colter in the first round as a quarterback. You got me.
Selection in First Round
If you’re looking for the best way to avoid being stuck with a Kyle Boller, it’s pretty simple: pick him early. And I mean very early.
Of the players in the top tier, the median pick where the teams drafted each respective player was 2.5. Only five of the 18 players (Rodgers, Daunte Culpepper, Ben Roethlisberger, Chad Pennington and Jay Cutler) were selected outside of the top 10. Seven of the 18 players were taken first overall, and 12 were top-four selections. In other words, when the scouts are convinced you’re a star, they’re generally right.
The median selection for the middle tier was eighth; the median for the bottom tier was 10th. Of course, it’s easy to point to guys like Tim Couch and David Carr, but they are the exception rather than the norm. The same goes for guys taken later in the draft like Rodgers – the success rate at the bottom is minuscule compared to that at the top.
Let’s break each of the 47 selections into two separate groups: those taken with the sixth pick or higher and those taken seventh or later. In total, there were 24 players taken sixth or higher. Of those, 13 were in the top tier, three were in the middle one, and eight were in the bottom tier. That’s a 54 percent hit rate (percentage of players that were in the top tier). This means that there were 23 players taken seventh overall or later in the first round. Five of those 22 were in the top tier, seven were in the middle tier, and 11 were in the bottom tier. That’s a 23 percent hit rate. The difference between the two is staggering.
This means that, for the most part, the scouts get it right. The players at the top of the draft are generally sure-fire prospects while those that slip to the teens have more glaring flaws.
As a stats geek, it hurts me to say this, but it’s hard to predict how a collegiate quarterback prospect will fare based on his college numbers. But that doesn’t mean they can’t help us make better-informed estimations. Players that don’t have as much experience present a greater risk. Those with a phenomenal Passer Efficiency are more likely to be a success than a bust. And you had better draft your franchise quarterback early, or you are flirting with spending the next five years chilling with Tim Tebow and Brady Quinn in the NFL Draft’s version of hell.