  Unsupervised meta-learning aims to learn feature representations from
unsupervised datasets that can transfer to downstream tasks with limited
labeled data. In this paper, we propose a novel approach to unsupervised
meta-learning that leverages the generalization abilities of in-context
learning observed in transformer architectures. Our method reframes
meta-learning as a sequence modeling problem, enabling the transformer encoder
to learn task context from support images and utilize it to predict query
images. At the core of our approach lies the creation of diverse tasks
generated using a combination of data augmentations and a mixing strategy that
challenges the model during training while fostering generalization to unseen
tasks at test time. Experimental results on benchmark datasets, including
miniImageNet, CIFAR-fs, CUB, and Aircraft, showcase the superiority of our
approach over existing unsupervised meta-learning baselines, establishing it as
the new state-of-the-art in the field. Remarkably, our method achieves
competitive results with supervised and self-supervised approaches,
underscoring the efficacy of the model in leveraging generalization over
memorization.
