  In cooperative multi-agent reinforcement learning (MARL) settings, the
centralized training with decentralized execution (CTDE) becomes customary
recently due to the physical demand. However, the most dilemma is the
inconsistency of jointly-trained policies and individually-optimized actions.
In this work, we propose a novel value-based multi-objective learning approach,
named Tchebycheff value decomposition optimization (TVDO), to overcome the
above dilemma. In particular, a nonlinear Tchebycheff aggregation method is
designed to transform the MARL task into multi-objective optimal counterpart by
tightly constraining the upper bound of individual action-value bias. We
theoretically prove that TVDO well satisfies the necessary and sufficient
condition of individual global max (IGM) with no extra limitations, which
exactly guarantees the consistency between the global and individual optimal
action-value function. Empirically, in the climb and penalty game, we verify
that TVDO represents precisely from global to individual value factorization
with a guarantee of the policy consistency. Furthermore, we also evaluate TVDO
in the challenging scenarios of StarCraft II micromanagement tasks, and
extensive experiments demonstrate that TVDO achieves more competitive
performances than several state-of-the-art MARL methods.
