  Communication in multi-agent reinforcement learning has been drawing
attention recently for its significant role in cooperation. However,
multi-agent systems may suffer from limitations on communication resources and
thus need efficient communication techniques in real-world scenarios. According
to the Shannon-Hartley theorem, messages to be transmitted reliably in worse
channels require lower entropy. Therefore, we aim to reduce message entropy in
multi-agent communication. A fundamental challenge is that the gradients of
entropy are either 0 or infinity, disabling gradient-based methods. To handle
it, we propose a pseudo gradient descent scheme, which reduces entropy by
adjusting the distributions of messages wisely. We conduct experiments on two
base communication frameworks with six environment settings and find that our
scheme can reduce message entropy by up to 90% with nearly no loss of
cooperation performance.
