  The field of cooperative multi-agent reinforcement learning (MARL) has seen
widespread use in addressing complex coordination tasks. While value
decomposition methods in MARL have been popular, they have limitations in
solving tasks with non-monotonic returns, restricting their general
application. Our work highlights the significance of joint intentions in
cooperation, which can overcome non-monotonic problems and increase the
interpretability of the learning process. To this end, we present a novel MARL
method that leverages learnable joint intentions. Our method employs a
hierarchical framework consisting of a joint intention policy and a behavior
policy to formulate the optimal cooperative policy. The joint intentions are
autonomously learned in a latent space through unsupervised learning and enable
the method adaptable to different agent configurations. Our results demonstrate
significant performance improvements in both the StarCraft micromanagement
benchmark and challenging MAgent domains, showcasing the effectiveness of our
method in learning meaningful joint intentions.
