  We examine the long-run behavior of multi-agent online learning in games that
evolve over time. Specifically, we focus on a wide class of policies based on
mirror descent, and we show that the induced sequence of play (a) converges to
Nash equilibrium in time-varying games that stabilize in the long run to a
strictly monotone limit; and (b) it stays asymptotically close to the evolving
equilibrium of the sequence of stage games (assuming they are strongly
monotone). Our results apply to both gradient-based and payoff-based feedback -
i.e., the "bandit feedback" case where players only get to observe the payoffs
of their chosen actions.
