  One of the key limitations of modern deep learning approaches lies in the
amount of data required to train them. Humans, by contrast, can learn to
recognize novel categories from just a few examples. Instrumental to this rapid
learning ability is the compositional structure of concept representations in
the human brain --- something that deep learning models are lacking. In this
work, we make a step towards bridging this gap between human and machine
learning by introducing a simple regularization technique that allows the
learned representation to be decomposable into parts. Our method uses
category-level attribute annotations to disentangle the feature space of a
network into subspaces corresponding to the attributes. These attributes can be
either purely visual, like object parts, or more abstract, like openness and
symmetry. We demonstrate the value of compositional representations on three
datasets: CUB-200-2011, SUN397, and ImageNet, and show that they require fewer
examples to learn classifiers for novel categories.
