  Deep neural networks achieve unprecedented performance levels over many tasks
and scale well with large quantities of data, but performance in the low-data
regime and tasks like one shot learning still lags behind. While recent work
suggests many hypotheses from better optimization to more complicated network
structures, in this work we hypothesize that having a learnable and more
expressive similarity objective is an essential missing component. Towards
overcoming that, we propose a network design inspired by deep residual networks
that allows the efficient computation of this more expressive pairwise
similarity objective. Further, we argue that regularization is key in learning
with small amounts of data, and propose an additional generator network based
on the Generative Adversarial Networks where the discriminator is our residual
pairwise network. This provides a strong regularizer by leveraging the
generated data samples. The proposed model can generate plausible variations of
exemplars over unseen classes and outperforms strong discriminative baselines
for few shot classification tasks. Notably, our residual pairwise network
design outperforms previous state-of-theart on the challenging mini-Imagenet
dataset for one shot learning by getting over 55% accuracy for the 5-way
classification task over unseen classes.
