  We assess the tendency of state-of-the-art object recognition models to
depend on signals from image backgrounds. We create a toolkit for disentangling
foreground and background signal on ImageNet images, and find that (a) models
can achieve non-trivial accuracy by relying on the background alone, (b) models
often misclassify images even in the presence of correctly classified
foregrounds--up to 87.5% of the time with adversarially chosen backgrounds, and
(c) more accurate models tend to depend on backgrounds less. Our analysis of
backgrounds brings us closer to understanding which correlations machine
learning models use, and how they determine models' out of distribution
performance.
