  The success of gradient-based meta-learning is primarily attributed to its
ability to leverage related tasks to learn task-invariant information. However,
the absence of interactions between different tasks in the inner loop leads to
task-specific over-fitting in the initial phase of meta-training. While this is
eventually corrected by the presence of these interactions in the outer loop,
it comes at a significant cost of slower meta-learning. To address this
limitation, we explicitly encode task relatedness via an inner loop
regularization mechanism inspired by multi-task learning. Our algorithm shares
gradient information from previously encountered tasks as well as concurrent
tasks in the same task batch, and scales their contribution with meta-learned
parameters. We show using two popular few-shot classification datasets that
gradient sharing enables meta-learning under bigger inner loop learning rates
and can accelerate the meta-training process by up to 134%.
