Thursday, February 20, 2014

The NFL Draft: My Methodology on Scouting, Evaluating & Ranking Prospects

from NFL.com


Over the next day or so I’m going to be posting some pre-Combine draft primer material; a way-too-early and incomplete Big Board, some positional rankings, and probably a hilariously-way-way-way-too-early mock draft. The following blog post is a good primer for that primer. It outlines in general (when I need to get more specific I will) how I go about evaluating players and how I develop my rankings. It’s a fairly arbitrary process that has been in work for over a decade (and I’m only 22!!...Wow I was a weird, awesome kid…) and is still continuing to evolve. I don’t get paid to be a “draft expert,” but I call myself one anyway. Links to previous years' rankings are at the very bottom of this post.

I’m not a professional scout. Nor have I ever worked as a professional scout. Nor have I ever been a professional coach or player. Nor have I ever been an amateur coach or player. I have absolutely no qualifications as a draft evaluator. Zero.

That said, for an “armchair GM” as such fellows as myself are called, I’m pretty damn good at it.

So how do I come up with my rankings?

It’s mostly done through excessive—almost absurd amounts—of reading. I read the obvious places: NFL.com, ESPN.com, CBSsports.com, BleacherReport.com, Scout.com, Rivals.com, WalterFootball.com. I read the obvious people: Mel Kiper, Mike Mayock, Daniel Jeremiah, Todd McShay, Gil Brandt, Greg Cosell, Rob Rang, Charley Casserly, Pat Kirwan, Matt Miller, Dane Brugler.

I look at less obvious websites that I find through extensive google searches. I read beat reporters. I read student newspapers if I have to. I just dig up information.

I read the scouting reports. I read the rumors. I read about work ethic and off-the-field issues. I read about which teams value certain players and where they value them, as reported by “sources” and “anonymous GMs.”

I watch the Senior Bowl. I watch the NFL Combine. I watch pro-days. I watch Gruden’s QB Camp. I watch Mel Kiper and Todd McShay yell at each other until spit flies from their lips and Mel’s forehead turns a shade of red particular to his angry face and unfortunate sunburn victims.

I watch as much college football as I can stomach. I don’t like the college game. I like the NFL. But I love, love, love the draft, so I tune in on Saturdays to see what I can see. Anytime I have an extremely high opinion of a player—a draft “crush,” if you will—it’s because I’ve seen them play live, and I thought, now that kid is a football player. I’ll get into this more later.

I also look at a player’s collegiate production and take into account the quality of their competition. This is the easiest part of the process, but one that factors in significantly.

I understand the “prototypes” for every position. I know what they should weigh, how tall they should be, how long their arms should be, how fast they should run the 40 yard dash, how many bench-press reps they can do, how high they can jump and how far they can leap. I know how big their hands are supposed to be.

Football isn’t played on paper, so height/weight/speed isn’t a tell-all, but it’s an important part of the scouting process. In fact, it makes up nearly one-third of it; both theoretically (meaning the majority of an evaluation should be based on game-play or film) and temporally (meaning the college season is typically approximately six months, and “draft season” is typically three).

I would be lying if I said I didn’t put a lot of stock into measurables. I try not to, but when you have limited access to game film and limited access to the players themselves for interviews and the like (i.e. none), you make do with what you do have.

Part of my methodology also comes from my bias as a 49ers fan, a bias I’m not ashamed of and never try to hide. I don’t necessarily order my Big Board rankings as if I were picking for the 49ers, but players I would like to see in red and gold inevitably sneak their way into and up my Top 100. I’m usually vocal about which players those are, and I usually let it be known when a players is being “knocked” for his not fitting in our scheme.

I then take all of these things into account, these varying components, and synthesize them into my own evaluation of a player. It's a mish-mash of personal observation, hearsay, and statistical analysis, with an ample dash of gut feeling. And I'm a master chef.

For example:

Last year Sharrif Floyd was widely considered the consensus top defensive tackle in the draft. He was rarely mocked outside the top-10 (usually he didn’t get past Oakland at No. 3), and even less rarely mocked or ranked behind Sheldon Richardson or Star Lotulelei.

So why did I have him below both those guys?

First of all, I watched all three of them play and I was taken with Sheldon Richardson right away. He wasn’t always dominant, but he was always impactful and always hustling. Star Lotulelei made almost the same impression. Watching Sharrif, I didn’t see the same impact. Obviously other draft evaluators did, but I didn’t.

Secondly, both Sheldon Richardson and Star Lotulelei are more physically and athletically imposing in measurable terms. While Star didn’t participate in many of the Combine drills (if my memory serves), Sheldon Richardson shined, confirming my opinion of him as an elite athlete. Elite athletes with high motors and great production (which Richardson has all of) come around fairly rarely, so I knew he was going to be special and I ranked him thusly. Sharrif Floyd has T-Rex syndrome (something Trent Baalke by and large does not tolerate). His arms are shorter than the “prototype.” It’s not always a big deal, but it does matter, especially when playing DT in a 4-3 against the run. It’s difficult to gain leverage. Many evaluators were quick to point it out, but more quick to dismiss it as an issue. I was not, probably because I hadn’t seen enough game tape to be enthralled by his play and feel like he worked around his lack of arm length effectively.

Thirdly, I wanted Sheldon Richardson to be a 49er. I wanted him to be the heir apparent to Justin Smith. He got a bump in ranking not only for that bias, but also because I loved his scheme versatility. He had enough length and strength to be a dominant five-technique (a DE in a 3-4 scheme), but was quick and fluid enough to play a penetrative three-technique (a DT in a 4-3). Star Lotulelei offered similar scheme versatility, and was similarly rewarded in my rankings. In fact, he was rewarded more so, because he had the size and agility to effectively (if not dominantly) man all three defensive tackle techniques; the five, the three and the zero (nose tackle).

So in the end I had Sheldon Richardson as my top-rated defensive player, Star Lotulelei as my third and Sharrif Floyd as my 11th  rated defensive prospect (my third DT). [Which, I should mention again, you can see in my 2013 Big Board and Positional Rankings, both linked at the bottom of this blog post.]

So far I look great on this (the exact reason I picked this as an example of my methodology in action) as Richardson was Defensive Rookie of the Year and Lotulelei wasn’t far off, while Floyd didn’t come close to living up to the hype surrounding him pre-draft (though in fairness I think injury had much to do with it).

However, as often as I look great I make mistakes. And I’ll call those out too.

I had Eric Reid rated 41st overall (41st!) and merely 4th amongst safeties.

While it’s way too early to judge the 2013 class, I think it’s fair to say Reid has justified or surpassed the expectations for him as the 18th overall pick and second safety taken off the board. I missed on scouting him, even in a year when I knew the 49ers would be looking to draft a safety to replace DaShon Goldson.

Looking back on it, I don’t have many excuses. Reid fits the bill for a prototype safety from a height/weight/speed perspective. I watched him make a ridiculous and game-changing interception of A.J. McCarron in LSU and Alabama’s initial meeting in the 2011 season. I knew he was smart, hard-working, and motivated from multiple reports. I knew Jim Harbaugh had tried to recruit him while he was at Stanford thanks to the reporting of the 49ers’ beat reporters.

I think that ultimately it came down to me buying into the hype, something I always try to avoid doing (and succeed at fairly well if I can pat myself on the back). As good as Reid was last year, it’s hard to argue without a lot of hindsight that he was a better prospect than Kenny Vaccaro, so I get a pass there. I think I fell in love with Cyprien and Swearinger in the draft process (mostly during the Senior Bowl) and merely bumped them ahead of Reid who I felt was in the same general class as them, but not quite as good. Cyprien had a ton of buzz around him because he was a big-time prospect coming out of a small school, which always gets a ton of reporting and internet scouting.

And there were a lot of scouting reports knocking Reid’s play in his final season of college as a junior, which many onlookers (the people I read) felt was worse than his breakout sophomore year. Much of it was attributed to the defensive talent which left LSU for the NFL Draft in between those two seasons. Reid’s ability to be a key playmaker in a defense was challenged. It was argued he needed strong pieces around him to succeed.

Whether or not that is the case I suppose time might tell. But it may not, because right now and in the foreseeable future the 49ers have plenty of talent on defense, so Reid may never have to be one of the lone “stars” on that side of the ball. It could be a B.S. knock on a great player anyway. Who knows? All I know is that he was great in his rookie season and far outperformed my fairly low ranking of him.

I can make excuses and take credit for other parts of my rankings—like ranking Trent Richardson second overall in 2012 (Jesus), or Russell Wilson 100th (but who didn’t underrate that guy), or ranking Jamie Collins 14th overall last year (man I’m good), or T.Y. Hilton 69th a couple years ago (he was pick 92nd…man I’m good)—which I have, for the first time ever, made available for public consumption on the internet.

I suppose there isn’t any way I can 100% guarantee I haven’t tampered with my rankings since the drafts they’re based on took place, but I can give you my word that I haven’t, if that means anything. These are no different than they were on the day of each of their respective drafts. As some sort of evidence there are a couple photos below that are taken of hard copies I printed out prior to each draft. 

The boxes I’ve filled in with hastily scrawled pen were left blank for that purpose, as I mark down when and by which team each player on my Big Board was drafted. It’s a good way for me to keep up with the draft as it happens live—a tradition I have completed each year for the past decade or more. Sadly, I don’t have my oldest rankings (I’ve been doing this annual filling out of the final Big Boards for at least five years, but I think longer). They’re in a box somewhere, or on an old computer or external hard drive long since gone. I do have 2012 and 2013 though, so you’re in luck.


I think you'd have to grow up a diehard 49er fan in the dark ages (the Erickson/Nolan/Singletary era) to develop this kind of obsession with the drafting of amateur athletes into a professional league. For almost 10 years it was the only thing us Niner fans really had to look forward to. Thankfully that's over now, but the obsession remains.

I’ve actually taken the time (and believe me it took some damn time) to fill-in the Big Boards on MS Word with the actual spot each player was drafted so you can conveniently peruse them and judge my efforts if you so please.

Links are at the bottom in the form of google docs.

PLEASE REMEMBER (sorry for yelling, this is important) the Big Board is not a mock draft. It is the order I would draft players based on their talent-level and projected production and contribution to a team, regardless of positional need. Basically, the best player in the draft is first, the second best second, etc. Sometimes (it’s rare) I leave players off of my Big Board even if they belong in the Top-100. I’m not an actual GM making actual picks, so usually guys with bad injuries or character concerns are left on the board, but it has happened that a player gets left off completely.

Also note that while it is nice to “hit” on where a player is drafted (e.g. a guy I have ranked 20th is picked at No. 20 in the actual draft) that’s also not really what this is about. That’s a good indication I had a similar read on the talent coming out of class as an actual NFL GM, but if my board is perfect (which is impossible), it doesn’t matter how many times I “hit.” It just matters that in three or four years you look at my rankings and think, yep that is the exact order, from highest to lowest in terms of how good each player actually has been in the NFL, that it should be in. So retrospectively Jerry Rice and Joe Montana should be at the top of the Big Boards from their draft classes.

Like I said, this is impossible. Literally. There are too many variables; injuries, coaching changes, scheme fit and scheme versatility, personal issues, etc. But it’s the goal. It’s also very difficult to judge positional value (a QB is more valuable than a guard, but how do you rank them in respect to one another if the guard is a better guard than a quarterback is a quarterback?).

It’s an imperfect process; an exercise in futility actually. Some of it is more art and instinct than science and evaluation. But it’s fun, and there is something to this whole scouting thing. It isn’t a complete crapshoot.

So, tomorrow I’ll start posting this year’s pre-Combine draft primer. I’ll have some initial and too-early positional rankings, offensive and defensive rankings, a not-fully-complete half-assed Big Board, and just for shits and giggles, a mock draft. I’ll also have some sparse analysis of certain players thrown in here and there.

The positional rankings and Big Board will change shortly after the Combine and will continue to grow and evolve during the months after the Combine leading up to the Draft on May 8th. So take everything I post now with a grain of salt.

Still, this is also an important part of the process—to develop early rankings now, before the Combine—so you can see how Combine performance, player interviews, and the media hype machine effects player perception throughout the draft process. Some of it is honing down on players that were previously underappreciated. Some of it is total bullshit. These rankings, in theory, should give us a good starting point from which to judge such developments.

It’s also a good way to get to know some names and familiarize yourself with the position groups to start developing your own “crushes.” Between half-a-dozen and a dozen of these guys are going to be in red and gold come May. Time to start doing your homework.


Links to Past Rankings:


No comments:

Post a Comment