  The stochastic compositional minimax problem has attracted a surge of
attention in recent years since it covers many emerging machine learning
models. Meanwhile, due to the emergence of distributed data, optimizing this
kind of problem under the decentralized setting becomes badly needed. However,
the compositional structure in the loss function brings unique challenges to
designing efficient decentralized optimization algorithms. In particular, our
study shows that the standard gossip communication strategy cannot achieve
linear speedup for decentralized compositional minimax problems due to the
large consensus error about the inner-level function. To address this issue, we
developed a novel decentralized stochastic compositional gradient descent
ascent with momentum algorithm to reduce the consensus error in the inner-level
function. As such, our theoretical results demonstrate that it is able to
achieve linear speedup with respect to the number of workers. We believe this
novel algorithmic design could benefit the development of decentralized
compositional optimization. Finally, we applied our methods to the imbalanced
classification problem. The extensive experimental results provide evidence for
the effectiveness of our algorithm.
