  Learning to optimize - the idea that we can learn from data algorithms that
optimize a numerical criterion - has recently been at the heart of a growing
number of research efforts. One of the most challenging issues within this
approach is to learn a policy that is able to optimize over classes of
functions that are fairly different from the ones that it was trained on. We
propose a novel way of framing learning to optimize as a problem of learning a
good navigation policy on a partially observable loss surface. To this end, we
develop Rover Descent, a solution that allows us to learn a fairly broad
optimization policy from training on a small set of prototypical
two-dimensional surfaces that encompasses the classically hard cases such as
valleys, plateaus, cliffs and saddles and by using strictly zero-order
information. We show that, without having access to gradient or curvature
information, we achieve state-of-the-art convergence speed on optimization
problems not presented at training time such as the Rosenbrock function and
other hard cases in two dimensions. We extend our framework to optimize over
high dimensional landscapes, while still handling only two-dimensional local
landscape information and show good preliminary results.
