  We develop a flexible stochastic approximation framework for analyzing the
long-run behavior of learning in games (both continuous and finite). The
proposed analysis template incorporates a wide array of popular learning
algorithms, including gradient-based methods, the exponential/multiplicative
weights algorithm for learning in finite games, optimistic and bandit variants
of the above, etc. In addition to providing an integrated view of these
algorithms, our framework further allows us to obtain several new convergence
results, both asymptotic and in finite time, in both continuous and finite
games. Specifically, we provide a range of criteria for identifying classes of
Nash equilibria and sets of action profiles that are attracting with high
probability, and we also introduce the notion of coherence, a game-theoretic
property that includes strict and sharp equilibria, and which leads to
convergence in finite time. Importantly, our analysis applies to both
oracle-based and bandit, payoff-based methods - that is, when players only
observe their realized payoffs.
