  Learning with limited data is a key challenge for visual recognition. Many
few-shot learning methods address this challenge by learning an instance
embedding function from seen classes and apply the function to instances from
unseen classes with limited labels. This style of transfer learning is
task-agnostic: the embedding function is not learned optimally discriminative
with respect to the unseen classes, where discerning among them leads to the
target task. In this paper, we propose a novel approach to adapt the instance
embeddings to the target classification task with a set-to-set function,
yielding embeddings that are task-specific and are discriminative. We
empirically investigated various instantiations of such set-to-set functions
and observed the Transformer is most effective -- as it naturally satisfies key
properties of our desired model. We denote this model as FEAT (few-shot
embedding adaptation w/ Transformer) and validate it on both the standard
few-shot classification benchmark and four extended few-shot learning settings
with essential use cases, i.e., cross-domain, transductive, generalized
few-shot learning, and low-shot learning. It archived consistent improvements
over baseline models as well as previous methods and established the new
state-of-the-art results on two benchmarks.
