  We study the limiting behavior of the mixed strategies that result from
optimal no-regret learning strategies in a repeated game setting where the
stage game is any 2 by 2 competitive game. We consider optimal no-regret
algorithms that are mean-based and monotonic in their argument. We show that
for any such algorithm, the limiting mixed strategies of the players cannot
converge almost surely to any Nash equilibrium. This negative result is also
shown to hold under a broad relaxation of these assumptions, including popular
variants of Online-Mirror-Descent with optimism and/or adaptive step-sizes.
Finally, we conjecture that the monotonicity assumption can be removed, and
provide partial evidence for this conjecture. Our results identify the inherent
stochasticity in players' realizations as a critical factor underlying this
divergence in outcomes between using the opponent's mixtures and realizations
to make updates.
