Almost exactly a year ago the Panthers’ 2018 offseason began, and with that came The Riot Report’s draft coverage – the ‘Scout Camp’ series – and before the 2019 offseason gets fully into gear, now seems like a suitable time to look back on the evaluations given in the 2018 series. The purpose of this piece then is to examine where I – Hi, I’m Vincent Richardson and I head up The Riot Report’s Draft Coverage – was wrong, where I was right, and where it’s still far too early to tell.
As an aside, draft evaluation is not only time-consuming – I watch about 200 hours of college football so you don’t have to – but it’s also extremely difficult and frustrating – read this article for more details on how just over 50% of first-round draft picks work out, but every single one is expected to be a cornerstone of your franchise:
Before we examine where things went well and what went badly, it is worth noting the methods employed in the evaluation. The evaluations are conducted using publicly available game film – almost all publications are not granted access to the college coaching tape used by teams or to the extended interviews with players that the NFL teams conduct. Additionally, the evaluations given did not take into account off the field concerns or medical history; I am not a lawyer, a behavioral psychologist nor a doctor. Now that that’s been said, how well did my pre-draft evaluations line up with the post-draft production?
The Good
Justin Reid. Photo Credit: Kelvin Kuo-USA TODAY Sports
It’s hard to be fully impartial when judging your own success, but I think it is fair to say that on the whole my evaluations have aged fairly well to this point. Some of the most successful mid and late round picks graded higher with us than with most other outlets and even most teams – Justin Reid and Mark Andrews are both seen as third round steals and both were in my top-15 prospects.
While we were far from alone on being high on these players, many of the best top-50 players also graded extremely well as well. Vita Vea, Bradley Chubb, Minkah Fitzpatrick, Marcus Davenport, Saquon Barkley, Nick Chubb, Da’Ron Payne, Calvin Ridley, Derwin James, Connor Williams and DJ Moore all graded as top 25 prospects, with Frank Ragnow, Dallas Goedert, Leighton Vander Esch and Will Hernandez all making our top 32.
On the other side of the coin, we also did well in avoiding some of the bigger early-round busts. Apparent busts such as Hayden Hurst and players who haven’t played up to their draft value – in the small sample size that was their rookie season – such as Kolton Miller and Billy Price, we viewed more apprehensively. There is obviously not time here to go after every pick in detail, but all of our positional draft boards are available on our website and you can make your own decision on how accurate we were over a range of positions – the search bar is above – try ‘Scout Camp’ and the position of your choosing; the last thing I’m doing is hiding from my opinions.
However, I certainly didn’t get everything right, so what did I get wrong and how can I be better in 2019?
Problematic Positions
There are a whole range of reasons why mistakes are made in evaluation, some are the result of insufficient information, some are the result of systematic method errors, some are simple errors of judgement and some are the result of subconscious biases. When a single player at a position is evaluated poorly, this is generally an indication of one of the latter two – or in rare cases the first of the four, whereas when an entire position is judged poorly, that tends to suggest that either a lack of information or a systematic methodical error is at fault. Two positions where my evaluations showed widespread inaccuracies were quarterback and cornerback, so let’s start with these.
Quarterback
As described in this 2018 Quarterback piece, evaluating quarterbacks tends to come down to two questions: do they have the arm to make the throws needed to play quarterback in the NFL and how likely are they do produce on that potential? Going into the draft, there were ten quarterback who I believed had an arm which gave them a chance of succeeding in the NFL: Rosen, Benkert, Darnold, Barrett, Falk, Rudolph, Allen, Jackson, Wolford and Mayfield – this is an assessment that I believe to have been successful.
The issue that then arises is how do you judge how likely those players are to develop into starting-caliber NFL quarterbacks, a question that then boils down to how effective they are as passers beyond just arm strength, and whether they are able to operate effectively in an NFL offense. Falk and Barrett were dismissed for this reason, with doubt cast over Rudolph and Wolford. That left Rosen, Benkert, Darnold, Allen, Jackson and Mayfield. So far, so uncontentious.
None of these players operated in offenses which allow them to be fully evaluated on tape, and without access to All-22 tape or offensive playbooks, it is hard to know completely how effective their decision making is in college; this is where the errors begin to creep in. Do you err on the side of caution and assume the worst case scenario, do you assume the best or do you just aim somewhere in between? Uncertainly is a real issue in evaluating prospects, but is more relevant to quarterbacks than other positions, and for better or worse, my viewpoint has been to err on the side of caution, leading me to rank players like Mayfield and Jackson much lower than their production in 2018 would merit. On the flip side, Josh Rosen and Kurt Benkert both didn’t show any massive concerns on tape, but neither have justified the grades I awarded to this point.
I aim to learn from this, and part of what I plan to include in the 2019 Scout Camp is to offer not just an overall grade, but an idea of where they are and where they could be. I don’t believe that I missed anything significant in evaluating Mayfield or Jackson, but rather that I conflated uncertainty and inability.
This is something I plan to do better in 2019.
Cornerback
My cornerback big board from 2018 is something of a mess, and this is a position which I honestly don’t enjoy evaluating – without all-22 tape it is almost impossible to get a complete picture. There are ways in which I think my methods could be better; but trying to judge players who you can’t see all the way through every play is always going to be prone to errors, let alone the levels of uncertainty associated with trying to judge players based on college defenses which rarely use anywhere near the range of coverages seen in the NFL.
Of course, I will always try and do the best I can with what I have to work with, but I have to accept that my judgments in this regard are going to be incomplete – read more about how tough the cornerback evaluation is in the article above..
Sometimes You Miss
Of course, some mistakes are not due to larger patterns out of our hands, but simply to errors in judgement. These are harder to rationalize and eliminate as their reasons aren’t always clear, but it would be dishonest not to mention them here. It is too early to describe anybody as a complete bust – except maybe Hayden Hurst – but of my top 32 prospects, the only player who has played poorly over an extended sample size is Josh Rosen.
Kurt Benkert. Photo Credit: Mark Gormus/Richmond Times-Dispatch
I definitely overvalued some players – Kurt Benkert at Quarterback, KC McDermott at Tackle, Ogbonnia Okoronkwo at 3-4 Outside Linebacker, and DeShon Elliott as safety were all players we expected to make early impacts and haven’t. There are others such as Roc Thomas and Tim Settle who we expected to have the potential to develop into contributors and who haven’t seen the field yet, as well as those such as Guice and Wynn who lost their seasons to injury. If these players don’t make an impact down the road then that is a problem, but it is hard fault Settle for not starting over Da’ron Payne and Jonathan Allen in Washington or Wynn from not being an impact player on IR.
There are also those who have played well who I didn’t grade highly. Some, such as Philip Lindsey came out of nowhere, averaging a better yards-per-carry with the Broncos than he did in college; some, such as Da’Shawn Hand I was wrongly in line with the NFL consensus and others such as Darius Leonard, I liked the potential but was overly cautious about the immediate impact and others, such as Dante Pettis, I simply misjudged.
It happens.
Some of this we will try to aim to work on by adding more nuance to how we grade our players, trying to separate the ideas of potential, concerns and proven skills, but despite what some might claim, this really isn’t a science and there is always going to be a degree of subjectivity – subjectivity which opens the door to mistakes. There is also the fact that players continue to develop when they enter the pros and some develop skills that it would be unreasonable to expect them to, and others never take the steps one would hope for – ‘teachability’ continues to be the great unknown in all of our tape-based assessments.
Up Next: The Big “Mistake”
