  Many machine learning tasks can be formulated as a stochastic compositional
optimization (SCO) problem such as reinforcement learning, AUC maximization,
and meta-learning, where the objective function involves a nested composition
associated with an expectation. While a significant amount of studies has been
devoted to studying the convergence behavior of SCO algorithms, there is little
work on understanding their generalization, i.e., how these learning algorithms
built from training examples would behave on future test examples. In this
paper, we provide the stability and generalization analysis of stochastic
compositional gradient descent algorithms through the lens of algorithmic
stability in the framework of statistical learning theory. Firstly, we
introduce a stability concept called compositional uniform stability and
establish its quantitative relation with generalization for SCO problems. Then,
we establish the compositional uniform stability results for two popular
stochastic compositional gradient descent algorithms, namely SCGD and SCSC.
Finally, we derive dimension-independent excess risk bounds for SCGD and SCSC
by trade-offing their stability results and optimization errors. To the best of
our knowledge, these are the first-ever-known results on stability and
generalization analysis of stochastic compositional gradient descent
algorithms.
