  With a computationally efficient approximation of the second-order
information, natural gradient methods have been successful in solving
large-scale structured optimization problems. We study the natural gradient
methods for the large-scale decentralized optimization problems on Riemannian
manifolds, where the local objective function defined by the local dataset is
of a log-probability type. By utilizing the structure of the Riemannian Fisher
information matrix (RFIM), we present an efficient decentralized Riemannian
natural gradient descent (DRNGD) method. To overcome the communication issue of
the high-dimension RFIM, we consider a class of structured problems for which
the RFIM can be approximated by a Kronecker product of two low-dimension
matrices. By performing the communications over the Kronecker factors, a
high-quality approximation of the RFIM can be obtained in a low cost. We prove
that DRNGD converges to a stationary point with the best-known rate of
$\mathcal{O}(1/K)$. Numerical experiments demonstrate the efficiency of our
proposed method compared with the state-of-the-art ones. To the best of our
knowledge, this is the first Riemannian second-order method for solving
decentralized manifold optimization problems.
