  Classical geometric mechanics, including the study of symmetries, Lagrangian
and Hamiltonian mechanics, and the Hamilton-Jacobi theory, are founded on
geometric structures such as jets, symplectic and contact ones. In this paper,
we shall use a partly forgotten framework of second-order (or stochastic)
differential geometry, developed originally by L. Schwartz and P.-A. Meyer, to
construct second-order counterparts of those classical structures. These will
allow us to study symmetries of stochastic differential equations (SDEs), to
establish stochastic Lagrangian and Hamiltonian mechanics and their key
relations with second-order Hamilton-Jacobi-Bellman (HJB) equations. Indeed,
stochastic prolongation formulae will be derived to study symmetries of SDEs
and mixed-order Cartan symmetries. Stochastic Hamilton's equations will follow
from a second-order symplectic structure and canonical transformations will
lead to the HJB equation. A stochastic variational problem on Riemannian
manifolds will provide a stochastic Euler-Lagrange equation compatible with HJB
one and equivalent to the Riemannian version of stochastic Hamilton's
equations. A stochastic Noether's theorem will also follow. The inspirational
example, along the paper, will be the rich dynamical structure of
Schr\"odinger's problem in optimal transport, where the latter is also regarded
as a Euclidean version of hydrodynamical interpretation of quantum mechanics.
