According to the Data: Is Week-to-Week Consistency an Illusion?

Is Week-to-Week Consistency an Illusion?

Prior to the season, I had Minnesota Vikings wide receiver Percy Harvin ranked as my No. 4 overall wide receiver in PPR leagues. In addition to his game-breaking ability, one of the main reasons I ranked Harvin so high is that I consider him "slump-proof." The nature of Harvin's game is so versatile that defenses have a difficult time containing him; he can play outside, in the slot, at running back, or anywhere else you can imagine, catching screens or deep passes and running end-arounds or read-options.

The truth is that Harvin is one of a limited number of players that I consider to possess true week-to-week consistency. The majority of what we perceive as weekly consistency is simply an illusion based on a limited sample size of games.

Imagine that a group of 25 receivers all have a 50 percent chance of putting up respectable fantasy numbers in a given game (how you define "respectable" is irrelevant to this example). For each player, the odds of posting quality fantasy stats are no better or no worse than a coin flip. What kind of results would we expect?

Typically, you'd see around half of players post between seven and nine respectable games. Almost all of the remaining players would fall between four and 12 respectable games, with a few outliers having either an outstanding season of 13-plus big-time performances or a horrible season of three or fewer quality games.

Actually, the results would resemble the graph below, which is a series of 400 coin flips I just completed (which sadly takes more time than you'd think and probably says more about me than anything you'll read hereafter). I broke the coin flips down into sets of 16 (to represent each game in an NFL season), tracking the number of heads that came up during each trial of 16 flips.

You can see that just over half of the trials ended up with exactly seven, eight, or nine heads. That's to be expected. What many people don't anticipate are the trials that end up as outliers: just a couple or all but a few flips being heads.

Actually, on my first 16 flips, 14 of them were heads. Even knowing that the chances of heads coming up were (almost) 50 percent, I began to think I was a biased coin-flipper. As I continued to flip, however, the number of tails "caught up", i.e., regressed toward the mean, and all was well in the world of randomness.

The takeaway here is that, in any set of random (or near-random) data, we'll see lots of "abnormal" results. If you assign Calvin Johnson a 50 percent chance of going for 100 yards and a touchdown in any given game, he'll probably wind up with somewhere around eight games with such numbers. But there's also a solid chance that he'll appear to have either an unusually outstanding or a very poor year. With a 50 percent chance of 100 yards and a score in any game, Megatron is probably around as likely to have either five or 12 stellar games as he is to have exactly eight.

Because the number of games in an NFL season is so low, it's really easy to see patterns in data that aren't really there. Over the course of even a few NFL seasons, we'd expect some players to appear to have a huge degree of weekly consistency, even if consistency were completely random. Similarly, even with total randomness, a handful of players would appear to be "all-or-nothing" fantasy options without much consistency, when in reality they possess just as much consistency as the most reliable performers.

To track past rates of consistency, I looked at high-performing wide receivers over the past two seasons. Of receivers with more than 60 catches, I examined the top 25 and the bottom 25 in terms of yards-per-reception. My hypothesis was that, if weekly consistency exists and is fairly strong, we'd see players with low yards-per-catch totals have the most consistent play. Those players - think Wes Welker, Danny Amendola, Percy Harvin - typically have a much higher reception rate (and often times more targets) than the big-play threats like DeSean Jackson, Mike Wallace and Brandon Lloyd.

The fact is that the big-play receivers were actually slightly more likely to have consistent play than the low-YPC receivers. That's after I adjusted for stat totals (the top 25 in YPC averaged 1,189 yards and 8.96 TDs per season, compared to 942 yards and 5.12 TDs for the bottom 25 in YPC). On average, the big-play receivers posted games with at least 6.0 percent of their final yardage total just over 9.5 times per year. For the low-YPC players - the ones who many consider to be very consistent on a week-to-week basis - the number was just under 9.5. The results are close enough to conclude that weekly receiving consistency, at least in terms of YPC, doesn't exist.

Like I said to start the article, I think there are a handful of players who are truly consistent. Slot receivers in particular (the Harvin's and Victor Cruz's of the world) are more difficult to double-team than X or Z receivers, and thus less likely to be held down in any particular game (stats on that in a future article).

As a general rule of thumb, though, beware of claims that Player X is a "safe bet this week" or Player Y is "really inconsistent." It's really, really easy to find patterns in past data, but really, really difficult to use those patterns to predict future data. As philosophy majors (all 20 of us) like to say, labeling a player as "consistent" is often done "ex post facto," or after the fact. In reality, weekly performances are a whole lot more random, and thus difficult to predict, than you might think.

Jonathan Bales is the author of Fantasy Football for Smart People: How to Dominate Your Draft. He also runs the "Running the Numbers" blog at DallasCowboys.com and writes for the New York Times.