  Few-shot learning is often motivated by the ability of humans to learn new
tasks from few examples. However, standard few-shot classification benchmarks
assume that the representation is learned on a limited amount of base class
data, ignoring the amount of prior knowledge that a human may have accumulated
before learning new tasks. At the same time, even if a powerful representation
is available, it may happen in some domain that base class data are limited or
non-existent. This motivates us to study a problem where the representation is
obtained from a classifier pre-trained on a large-scale dataset of a different
domain, assuming no access to its training process, while the base class data
are limited to few examples per class and their role is to adapt the
representation to the domain at hand rather than learn from scratch. We adapt
the representation in two stages, namely on the few base class data if
available and on the even fewer data of new tasks. In doing so, we obtain from
the pre-trained classifier a spatial attention map that allows focusing on
objects and suppressing background clutter. This is important in the new
problem, because when base class data are few, the network cannot learn where
to focus implicitly. We also show that a pre-trained network may be easily
adapted to novel classes, without meta-learning.
