  Reinforcement learning in heterogeneous multi-agent scenarios is important
for real-world applications but presents challenges beyond those seen in
homogeneous settings and simple benchmarks. In this work, we present an
actor-critic algorithm that allows a team of heterogeneous agents to learn
decentralized control policies for covering an unknown environment. This task
is of interest to national security and emergency response organizations that
would like to enhance situational awareness in hazardous areas by deploying
teams of unmanned aerial vehicles. To solve this multi-agent coverage path
planning problem in unknown environments, we augment a multi-agent actor-critic
architecture with a new state encoding structure and triplet learning loss to
support heterogeneous agent learning. We developed a simulation environment
that includes real-world environmental factors such as turbulence, delayed
communication, and agent loss, to train teams of agents as well as probe their
robustness and flexibility to such disturbances.
