  We provide an Information-Geometric formulation of Classical Mechanics on the
Riemannian manifold of probability distributions, which is an affine manifold
endowed with a dually-flat connection. In a non-parametric formalism, we
consider the full set of positive probability functions on a finite sample
space, and we provide a specific expression for the tangent and cotangent
spaces over the statistical manifold, in terms of a Hilbert bundle structure
that we call the Statistical Bundle. In this setting, we compute velocities and
accelerations of a one-dimensional statistical model using the canonical dual
pair of parallel transports and define a coherent formalism for Lagrangian and
Hamiltonian mechanics on the bundle. Finally, in a series of examples, we show
how our formalism provides a consistent framework for accelerated natural
gradient dynamics on the probability simplex, paving the way for direct
applications in optimization, game theory and neural networks.
