  This paper considers a time-varying game with $N$ players. Every time slot,
players observe their own random events and then take a control action. The
events and control actions affect the individual utilities earned by each
player. The goal is to maximize a concave function of time average utilities
subject to equilibrium constraints. Specifically, participating players are
provided access to a common source of randomness from which they can optimally
correlate their decisions. The equilibrium constraints incentivize
participation by ensuring that players cannot earn more utility if they choose
not to participate. This form of equilibrium is similar to the notions of Nash
equilibrium and correlated equilibrium, but is simpler to attain. A Lyapunov
method is developed that solves the problem in an online \emph{max-weight}
fashion by selecting actions based on a set of time-varying weights. The
algorithm does not require knowledge of the event probabilities and has
polynomial convergence time. A similar method can be used to compute a standard
correlated equilibrium, albeit with increased complexity.
