  Quarterback performance can be difficult to rank, and much effort has been
spent in creating new rating systems. However, the input statistics for such
ratings are subject to randomness and factors outside the quarterback's
control. To investigate this variance, we perform a sensitivity analysis of
three quarterback rating statistics: the Traditional 1971 rating by Smith, the
Burke, and the Wages of Wins ratings. The comparisons are made at the team
level for the 32 NFL teams from 2002-2015, thus giving each case an even 16
games. We compute quarterback ratings for each offense with 1-5 additional
touchdowns, 1-5 fewer interceptions, 1-5 additional sacks, and a 1-5 percent
increase in the passing completion rate. Our sensitivity analysis provides
insight into whether an elite passing team could seem mediocre or vice versa
based on random outcomes. The results indicate that the Traditional rating is
the most sensitive statistic with respect to touchdowns, interceptions, and
completions, whereas the Burke rating is most sensitive to sacks. The analysis
suggests that team passing offense rankings are highly sensitive to aspects of
football that are out of the quarterback's hands (e.g., deflected passes that
lead to interceptions). Thus, on the margins, we show arguments about whether a
specific quarterback has entered the elite or remains mediocre are irrelevant.
