  In Riemannian geometry geodesics are integral curves of the Riemannian
distance gradient. We extend this classical result to the framework of
Information Geometry. In particular, we prove that the rays of level-sets
defined by a pseudo-distance are generated by the sum of two tangent vectors.
By relying on these vectors, we propose a novel definition of a canonical
divergence and its dual function. We prove that the new divergence allows to
recover a given dual structure $(\mathrm{g},\nabla,\nabla^*)$ of {a dually
convex set on} a smooth manifold $\mathrm{M}$. Additionally, we show that this
divergence coincides with the canonical divergence proposed by Ay and Amari in
the case of: (a) self-duality, (b) dual flatness, (c) statistical geometric
analogue of the concept of symmetric spaces in Riemannian geometry. For a
dually convex set, the case (c) leads to a further comparison of the new
divergence with the one introduced by Henmi and Kobayashi.
