  We demonstrate the versatility of mean-field games (MFGs) as a mathematical
framework for explaining, enhancing, and designing generative models. In
generative flows, a Lagrangian formulation is used where each particle
(generated sample) aims to minimize a loss function over its simulated path.
The loss, however, is dependent on the paths of other particles, which leads to
a competition among the population of particles. The asymptotic behavior of
this competition yields a mean-field game. We establish connections between
MFGs and major classes of generative flows and diffusions including
continuous-time normalizing flows, score-based generative models (SGM), and
Wasserstein gradient flows. Furthermore, we study the mathematical properties
of each generative model by studying their associated MFG's optimality
condition, which is a set of coupled forward-backward nonlinear partial
differential equations. The mathematical structure described by the MFG
optimality conditions identifies the inductive biases of generative flows. We
investigate the well-posedness and structure of normalizing flows, unravel the
mathematical structure of SGMs, and derive a MFG formulation of Wasserstein
gradient flows. From an algorithmic perspective, the optimality conditions
yields Hamilton-Jacobi-Bellman (HJB) regularizers for enhanced training of
generative models. In particular, we propose and demonstrate an HJB-regularized
SGM with improved performance over standard SGMs. We present this framework as
an MFG laboratory which serves as a platform for revealing new avenues of
experimentation and invention of generative models.
