  This paper concerns imitation learning (IL) (i.e, the problem of learning to
mimic expert behaviors from demonstrations) in cooperative multi-agent systems.
The learning problem under consideration poses several challenges,
characterized by high-dimensional state and action spaces and intricate
inter-agent dependencies. In a single-agent setting, IL has proven to be done
efficiently through an inverse soft-Q learning process given expert
demonstrations. However, extending this framework to a multi-agent context
introduces the need to simultaneously learn both local value functions to
capture local observations and individual actions, and a joint value function
for exploiting centralized learning. In this work, we introduce a novel
multi-agent IL algorithm designed to address these challenges. Our approach
enables the centralized learning by leveraging mixing networks to aggregate
decentralized Q functions. A main advantage of this approach is that the
weights of the mixing networks can be trained using information derived from
global states. We further establish conditions for the mixing networks under
which the multi-agent objective function exhibits convexity within the Q
function space. We present extensive experiments conducted on some challenging
competitive and cooperative multi-agent game environments, including an
advanced version of the Star-Craft multi-agent challenge (i.e., SMACv2), which
demonstrates the effectiveness of our proposed algorithm compared to existing
state-of-the-art multi-agent IL algorithms.
