  The impressive success of modern deep neural networks on computer vision
tasks has been achieved through models of very large capacity compared to the
number of available training examples. This overparameterization is often said
to be controlled with the help of different regularization techniques, mainly
weight decay and dropout. However, since these techniques reduce the effective
capacity of the model, typically even deeper and wider architectures are
required to compensate for the reduced capacity. Therefore, there seems to be a
waste of capacity in this practice. In this paper we build upon recent research
that suggests that explicit regularization may not be as important as widely
believed and carry out an ablation study that concludes that weight decay and
dropout may not be necessary for object recognition if enough data augmentation
is introduced.
