  The minimax optimization over Riemannian manifolds (possibly nonconvex
constraints) has been actively applied to solve many problems, such as robust
dimensionality reduction and deep neural networks with orthogonal weights
(Stiefel manifold). Although many optimization algorithms for minimax problems
have been developed in the Euclidean setting, it is difficult to convert them
into Riemannian cases, and algorithms for nonconvex minimax problems with
nonconvex constraints are even rare. On the other hand, to address the big data
challenges, decentralized (serverless) training techniques have recently been
emerging since they can reduce communications overhead and avoid the bottleneck
problem on the server node. Nonetheless, the algorithm for decentralized
Riemannian minimax problems has not been studied. In this paper, we study the
distributed nonconvex-strongly-concave minimax optimization problem over the
Stiefel manifold and propose both deterministic and stochastic minimax methods.
The Steifel manifold is a non-convex set. The global function is represented as
the finite sum of local functions. For the deterministic setting, we propose
DRGDA and prove that our deterministic method achieves a gradient complexity of
$O( \epsilon^{-2})$ under mild conditions. For the stochastic setting, we
propose DRSGDA and prove that our stochastic method achieves a gradient
complexity of $O(\epsilon^{-4})$. The DRGDA and DRSGDA are the first algorithms
for distributed minimax optimization with nonconvex constraints with exact
convergence. Extensive experimental results on the Deep Neural Networks (DNNs)
training over the Stiefel manifold demonstrate the efficiency of our
algorithms.
