  The capacity of meta-learning algorithms to quickly adapt to a variety of
tasks, including ones they did not experience during meta-training, has been a
key factor in the recent success of these methods on few-shot learning
problems. This particular advantage of using meta-learning over standard
supervised or reinforcement learning is only well founded under the assumption
that the adaptation phase does improve the performance of our model on the task
of interest. However, in the classical framework of meta-learning, this
constraint is only mildly enforced, if not at all, and we only see an
improvement on average over a distribution of tasks. In this paper, we show
that the adaptation in an algorithm like MAML can significantly decrease the
performance of an agent in a meta-reinforcement learning setting, even on a
range of meta-training tasks.
