  We present two logit-Q learning dynamics combining the classical and
independent log-linear learning updates with an on-policy value iteration
update for efficient learning in stochastic games. We show that the logit-Q
dynamics presented reach (near) efficient equilibrium in stochastic teams. We
quantify a bound on the approximation error. We also show the rationality of
the logit-Q dynamics against agents following pure stationary strategies and
the convergence of the dynamics in stochastic games where the reward functions
induce potential games, yet only a single agent controls the state transitions
beyond stochastic teams. The key idea is to approximate the dynamics with a
fictional scenario where the Q-function estimates are stationary over
finite-length epochs only for analysis. We then couple the dynamics in the main
and fictional scenarios to show that these two scenarios become more and more
similar across epochs due to the vanishing step size.
