  While datasets with single-label supervision have propelled rapid advances in
image classification, additional annotations are necessary in order to
quantitatively assess how models make predictions. To this end, for a subset of
ImageNet samples, we collect segmentation masks for the entire object and $18$
informative attributes. We call this dataset RIVAL10 (RIch Visual Attributes
with Localization), consisting of roughly $26k$ instances over $10$ classes.
Using RIVAL10, we evaluate the sensitivity of a broad set of models to noise
corruptions in foregrounds, backgrounds and attributes. In our analysis, we
consider diverse state-of-the-art architectures (ResNets, Transformers) and
training procedures (CLIP, SimCLR, DeiT, Adversarial Training). We find that,
somewhat surprisingly, in ResNets, adversarial training makes models more
sensitive to the background compared to foreground than standard training.
Similarly, contrastively-trained models also have lower relative foreground
sensitivity in both transformers and ResNets. Lastly, we observe intriguing
adaptive abilities of transformers to increase relative foreground sensitivity
as corruption level increases. Using saliency methods, we automatically
discover spurious features that drive the background sensitivity of models and
assess alignment of saliency maps with foregrounds. Finally, we quantitatively
study the attribution problem for neural features by comparing feature saliency
with ground-truth localization of semantic attributes.
