  In this paper, we propose a learning algorithm that enables a model to
quickly exploit commonalities among related tasks from an unseen task
distribution, before quickly adapting to specific tasks from that same
distribution. We investigate how learning with different task distributions can
first improve adaptability by meta-finetuning on related tasks before improving
goal task generalization with finetuning. Synthetic regression experiments
validate the intuition that learning to meta-learn improves adaptability and
consecutively generalization. Experiments on more complex image classification,
continual regression, and reinforcement learning tasks demonstrate that
learning to meta-learn generally improves task-specific adaptation. The
methodology, setup, and hypotheses in this proposal were positively evaluated
by peer review before conclusive experiments were carried out.
