  This paper presents new families of algorithms for the repeated play of
two-agent (near) zero-sum games and two-agent zero-sum stochastic games. For
example, the family includes fictitious play and its variants as members.
Commonly, the algorithms in this family are all uncoupled, rational, and
convergent even in heterogeneous cases, e.g., where the dynamics may differ in
terms of learning rates, full, none or temporal access to opponent actions, and
model-based vs model-free learning. The convergence of heterogeneous dynamics
is of practical interest especially in competitive environments since agents
may have no means or interests in following the same dynamic with the same
parameters. We prove that any mixture of such asymmetries does not impact the
algorithms' convergence to equilibrium (or near equilibrium if there is
experimentation) in zero-sum games with repeated play and in zero-sum
(irreducible) stochastic games with sufficiently small discount factors.
