  We consider the fundamental task of optimising a real-valued function defined
in a potentially high-dimensional Euclidean space, such as the loss function in
many machine-learning tasks or the logarithm of the probability distribution in
statistical inference. We use Riemannian geometry notions to redefine the
optimisation problem of a function on the Euclidean space to a Riemannian
manifold with a warped metric, and then find the function's optimum along this
manifold. The warped metric chosen for the search domain induces a
computational friendly metric-tensor for which optimal search directions
associated with geodesic curves on the manifold becomes easier to compute.
Performing optimization along geodesics is known to be generally infeasible,
yet we show that in this specific manifold we can analytically derive Taylor
approximations up to third-order. In general these approximations to the
geodesic curve will not lie on the manifold, however we construct suitable
retraction maps to pull them back onto the manifold. Therefore, we can
efficiently optimize along the approximate geodesic curves. We cover the
related theory, describe a practical optimization algorithm and empirically
evaluate it on a collection of challenging optimisation benchmarks. Our
proposed algorithm, using 3rd-order approximation of geodesics, tends to
outperform standard Euclidean gradient-based counterparts in term of number of
iterations until convergence.
