  We study the connection between gradient-based meta-learning and convex
op-timisation. We observe that gradient descent with momentum is a special case
of meta-gradients, and building on recent results in optimisation, we prove
convergence rates for meta-learning in the single task setting. While a
meta-learned update rule can yield faster convergence up to constant factor, it
is not sufficient for acceleration. Instead, some form of optimism is required.
We show that optimism in meta-learning can be captured through Bootstrapped
Meta-Gradients (Flennerhag et al., 2022), providing deeper insight into its
underlying mechanics.
