  Conventional wisdom suggests that neural network predictions tend to be
unpredictable and overconfident when faced with out-of-distribution (OOD)
inputs. Our work reassesses this assumption for neural networks with
high-dimensional inputs. Rather than extrapolating in arbitrary ways, we
observe that neural network predictions often tend towards a constant value as
input data becomes increasingly OOD. Moreover, we find that this value often
closely approximates the optimal constant solution (OCS), i.e., the prediction
that minimizes the average loss over the training data without observing the
input. We present results showing this phenomenon across 8 datasets with
different distributional shifts (including CIFAR10-C and ImageNet-R, S),
different loss functions (cross entropy, MSE, and Gaussian NLL), and different
architectures (CNNs and transformers). Furthermore, we present an explanation
for this behavior, which we first validate empirically and then study
theoretically in a simplified setting involving deep homogeneous networks with
ReLU activations. Finally, we show how one can leverage our insights in
practice to enable risk-sensitive decision-making in the presence of OOD
inputs.
