  We study a simple model of algorithmic collusion in which Q-learning
algorithms repeatedly play a prisoner's dilemma and allow players to choose
different exploration policies. We characterize behavior of such algorithms
with asymmetric policies for extreme values and prove that any Nash equilibrium
features some cooperative behavior. We further investigate the dynamics for
general profiles of exploration policy by running extensive numerical
simulations which indicate symmetry of equilibria, and give insight for their
distribution.
