  Differential graphical models are designed to represent the difference
between the conditional dependence structures of two groups, thus are of
particular interest for scientific investigation. Motivated by modern
applications, this manuscript considers an extended setting where each group is
generated by a latent variable Gaussian graphical model. Due to the existence
of latent factors, the differential network is decomposed into sparse and
low-rank components, both of which are symmetric indefinite matrices. We
estimate these two components simultaneously using a two-stage procedure: (i)
an initialization stage, which computes a simple, consistent estimator, and
(ii) a convergence stage, implemented using a projected alternating gradient
descent algorithm applied to a nonconvex objective, initialized using the
output of the first stage. We prove that given the initialization, the
estimator converges linearly with a nontrivial, minimax optimal statistical
error. Experiments on synthetic and real data illustrate that the proposed
nonconvex procedure outperforms existing methods.
