  Recent works have demonstrated that neural networks exhibit extreme
simplicity bias(SB). That is, they learn only the simplest features to solve a
task at hand, even in the presence of other, more robust but more complex
features. Due to the lack of a general and rigorous definition of features,
these works showcase SB on semi-synthetic datasets such as Color-MNIST,
MNIST-CIFAR where defining features is relatively easier.
  In this work, we rigorously define as well as thoroughly establish SB for one
hidden layer neural networks. More concretely, (i) we define SB as the network
essentially being a function of a low dimensional projection of the inputs (ii)
theoretically, we show that when the data is linearly separable, the network
primarily depends on only the linearly separable ($1$-dimensional) subspace
even in the presence of an arbitrarily large number of other, more complex
features which could have led to a significantly more robust classifier, (iii)
empirically, we show that models trained on real datasets such as Imagenette
and Waterbirds-Landbirds indeed depend on a low dimensional projection of the
inputs, thereby demonstrating SB on these datasets, iv) finally, we present a
natural ensemble approach that encourages diversity in models by training
successive models on features not used by earlier models, and demonstrate that
it yields models that are significantly more robust to Gaussian noise.
