  We study optimization problems whereby the optimization variable is a
probability measure. Since the probability space is not a vector space, many
classical and powerful methods for optimization (e.g., gradients) are of little
help. Thus, one typically resorts to the abstract machinery of
infinite-dimensional analysis or other ad-hoc methodologies, not tailored to
the probability space, which however involve projections or rely on
convexity-type assumptions. We believe instead that these problems call for a
comprehensive methodological framework for calculus in probability spaces. In
this work, we combine ideas from optimal transport, variational analysis, and
Wasserstein gradient flows to equip the Wasserstein space (i.e., the space of
probability measures endowed with the Wasserstein distance) with a variational
structure, both by combining and extending existing results and introducing
novel tools. Our theoretical analysis culminates in very general necessary
optimality conditions for optimality. Notably, our conditions (i) resemble the
rationales of Euclidean spaces, such as the Karush-Kuhn-Tucker and Lagrange
conditions, (ii) are intuitive, informative, and easy to study, and (iii) yield
closed-form solutions or can be used to design computationally attractive
algorithms. We believe this framework lays the foundation for new algorithmic
and theoretical advancements in the study of optimization problems in
probability spaces, which we exemplify with numerous case studies and
applications to machine learning, drug discovery, and distributionally robust
optimization.
