  The aim of this paper is to study how to apply deep reinforcement learning
for the control of aggregates of minimalistic robots. We define aggregates as
groups of robots with a physical connection that compels them to form a
specified shape. In our case, the robots are pre-attached to an object that
must be collectively transported to a known location. Minimalism, in our
setting, stems from the barebone capabilities we assume: The robots can sense
the target location and the immediate obstacles, but lack the means to
communicate explicitly through, e.g., message-passing. In our setting,
communication is implicit, i.e., mediated by aggregated push-and-pull on the
object exerted by each robot. We analyze the ability to reach coordinated
behavior of four well-known algorithms for deep reinforcement learning (DQN,
DDQN, DDPG, and TD3). Our experiments include robot failures and different
types of environmental obstacles. We compare the performance of the best
control strategies found, highlighting strengths and weaknesses of each of the
considered training algorithms.
