  Value function factorization methods are commonly used in cooperative
multi-agent reinforcement learning, with QMIX receiving significant attention.
Many QMIX-based methods introduce monotonicity constraints between the joint
action value and individual action values to achieve decentralized execution.
However, such constraints limit the representation capacity of value
factorization, restricting the joint action values it can represent and
hindering the learning of the optimal policy. To address this challenge, we
propose the Potentially Optimal joint actions Weighted QMIX (POWQMIX)
algorithm, which recognizes the potentially optimal joint actions and assigns
higher weights to the corresponding losses of these joint actions during
training. We theoretically prove that with such a weighted training approach
the optimal policy is guaranteed to be recovered. Experiments in matrix games,
predator-prey, and StarCraft II Multi-Agent Challenge environments demonstrate
that our algorithm outperforms the state-of-the-art value-based multi-agent
reinforcement learning methods.
