  Recently, decentralized optimization over the Stiefel manifold has attacked
tremendous attentions due to its wide range of applications in various fields.
Existing methods rely on the gradients to update variables, which are not
applicable to the objective functions with non-smooth regularizers, such as
sparse PCA. In this paper, to the best of our knowledge, we propose the first
decentralized algorithm for non-smooth optimization over Stiefel manifolds. Our
algorithm approximates the non-smooth part of objective function by its Moreau
envelope, and then existing algorithms for smooth optimization can be deployed.
We establish the convergence guarantee with the iteration complexity of
$\mathcal{O} (\epsilon^{-4})$. Numerical experiments conducted under the
decentralized setting demonstrate the effectiveness and efficiency of our
algorithm.
