  While multi-agent interactions can be naturally modeled as a graph, the
environment has traditionally been considered as a black box. We propose to
create a shared agent-entity graph, where agents and environmental entities
form vertices, and edges exist between the vertices which can communicate with
each other. Agents learn to cooperate by exchanging messages along the edges of
this graph. Our proposed multi-agent reinforcement learning framework is
invariant to the number of agents or entities present in the system as well as
permutation invariance, both of which are desirable properties for any
multi-agent system representation. We present state-of-the-art results on
coverage, formation and line control tasks for multi-agent teams in a fully
decentralized framework and further show that the learned policies quickly
transfer to scenarios with different team sizes along with strong zero-shot
generalization performance. This is an important step towards developing
multi-agent teams which can be realistically deployed in the real world without
assuming complete prior knowledge or instantaneous communication at unbounded
distances.
