  The standard mathematical approach to fourth-down decision making in American
football is to make the decision that maximizes estimated win probability. Win
probability estimates arise from machine learning models fit from historical
data. These models attempt to capture a nuanced relationship between a noisy
binary outcome variable and game-state variables replete with interactions and
non-linearities from a finite dataset of just a few thousand games. Thus, it is
imperative to knit uncertainty quantification into the fourth-down decision
procedure; we do so using bootstrapping. We find that uncertainty in the
estimated optimal fourth-down decision is far greater than that currently
expressed by sports analysts in popular sports media.
