  To improve the performance of multi-agent reinforcement learning under the
constraint of wireless resources, we propose a message importance metric and
design an importance-aware scheduling policy to effectively exchange messages.
The key insight is spending the precious communication resources on important
messages. The message importance depends not only on the messages themselves,
but also on the needs of agents who receive them. Accordingly, we propose a
query-message-based architecture, called QMNet. Agents generate queries and
messages with the environment observation. Sharing queries can help calculate
message importance. Exchanging messages can help agents cooperate better.
Besides, we exploit the message importance to deal with random access
collisions in decentralized systems. Furthermore, a message prediction
mechanism is proposed to compensate for messages that are not transmitted.
Finally, we evaluate the proposed schemes in a traffic junction environment,
where only a fraction of agents can send messages due to limited wireless
resources. Results show that QMNet can extract valuable information to
guarantee the system performance even when only $30\%$ of agents can share
messages. By exploiting message prediction, the system can further save $40\%$
of wireless resources. The importance-aware decentralized multi-access
mechanism can effectively avoid collisions, achieving almost the same
performance as centralized scheduling.
