  We consider solving a cooperative multi-robot object manipulation task using
reinforcement learning (RL). We propose two distributed multi-agent RL
approaches: distributed approximate RL (DA-RL), where each agent applies
Q-learning with individual reward functions; and game-theoretic RL (GT-RL),
where the agents update their Q-values based on the Nash equilibrium of a
bimatrix Q-value game. We validate the proposed approaches in the setting of
cooperative object manipulation with two simulated robot arms. Although we
focus on a small system of two agents in this paper, both DA-RL and GT-RL apply
to general multi-agent systems, and are expected to scale well to large
systems.
