  We show that the principle of maximum entropy, a variational method appearing
in statistical inference, statistical physics, and the analysis of stochastic
dynamical systems, admits a geometric description from gauge theory. Using the
connection on a principal $G$-bundle, the gradient flow maximising entropy is
written in terms of constraint functions which interact with the dynamics of
the probabilistic degrees of freedom of a diffusion process. This allows us to
describe the point of maximum entropy as parallel transport over the state
space. In particular, it is proven that the solubility of the stationary
Fokker--Planck equation corresponds to the existence of parallel transport in a
particular associated vector bundle, extending classic results due to
Jordan--Kinderlehrer--Otto and Markowich--Villani. A reinterpretation of
splitting results in stochastic dynamical systems is also suggested. Beyond
stochastic analysis, we are able to indicate a collection of geometric
structures surrounding energy-based inference in statistics.
