  Deep learning has shown that learned functions can dramatically outperform
hand-designed functions on perceptual tasks. Analogously, this suggests that
learned optimizers may similarly outperform current hand-designed optimizers,
especially for specific problems. However, learned optimizers are notoriously
difficult to train and have yet to demonstrate wall-clock speedups over
hand-designed optimizers, and thus are rarely used in practice. Typically,
learned optimizers are trained by truncated backpropagation through an unrolled
optimization process resulting in gradients that are either strongly biased
(for short truncations) or have exploding norm (for long truncations). In this
work we propose a training scheme which overcomes both of these difficulties,
by dynamically weighting two unbiased gradient estimators for a variational
loss on optimizer performance, allowing us to train neural networks to perform
optimization of a specific task faster than tuned first-order methods. We
demonstrate these results on problems where our learned optimizer trains
convolutional networks faster in wall-clock time compared to tuned first-order
methods and with an improvement in test loss.
