  Few-shot classification refers to learning a classifier for new classes given
only a few examples. While a plethora of models have emerged to tackle it, we
find the procedure and datasets that are used to assess their progress lacking.
To address this limitation, we propose Meta-Dataset: a new benchmark for
training and evaluating models that is large-scale, consists of diverse
datasets, and presents more realistic tasks. We experiment with popular
baselines and meta-learners on Meta-Dataset, along with a competitive method
that we propose. We analyze performance as a function of various
characteristics of test tasks and examine the models' ability to leverage
diverse training sources for improving their generalization. We also propose a
new set of baselines for quantifying the benefit of meta-learning in
Meta-Dataset. Our extensive experimentation has uncovered important research
challenges and we hope to inspire work in these directions.
