  Starting with a group of reinforcement-learning agents we derive coupled
replicator equations that describe the dynamics of collective learning in
multiagent systems. We show that, although agents model their environment in a
self-interested way without sharing knowledge, a game dynamics emerges
naturally through environment-mediated interactions. An application to
rock-scissors-paper game interactions shows that the collective learning
dynamics exhibits a diversity of competitive and cooperative behaviors. These
include quasiperiodicity, stable limit cycles, intermittency, and deterministic
chaos--behaviors that should be expected in heterogeneous multiagent systems
described by the general replicator equations we derive.
