  We consider communication in a fully cooperative multi-agent system, where
the agents have partial observation of the environment and must act jointly to
maximize the overall reward. We have a discrete-time queueing network where
agents route packets to queues based only on the partial information of the
current queue lengths. The queues have limited buffer capacity, so packet drops
happen when they are sent to a full queue. In this work, we implemented a
communication channel for the agents to share their information in order to
reduce the packet drop rate. For efficient information sharing we use an
attention-based communication model, called ATVC, to select informative
messages from other agents. The agents then infer the state of queues using a
combination of the variational auto-encoder, VAE, and product-of-experts, PoE,
model. Ultimately, the agents learn what they need to communicate and with
whom, instead of communicating all the time with everyone. We also show
empirically that ATVC is able to infer the true state of the queues and leads
to a policy which outperforms existing baselines.
