  We study non-convex distributed optimization problems where a set of agents
collaboratively solve a separable optimization problem that is distributed over
a time-varying network. The existing methods to solve these problems rely on
(at most) one time-scale algorithms, where each agent performs a diminishing or
constant step-size gradient descent at the average estimate of the agents in
the network. However, if possible at all, exchanging exact information, that is
required to evaluate these average estimates, potentially introduces a massive
communication overhead. Therefore, a reasonable practical assumption to be made
is that agents only receive a rough approximation of the neighboring agents'
information. To address this, we introduce and study a \textit{two time-scale}
decentralized algorithm with a broad class of \textit{lossy} information
sharing methods (that includes noisy, quantized, and/or compressed information
sharing) over \textit{time-varying} networks. In our method, one time-scale
suppresses the (imperfect) incoming information from the neighboring agents,
and one time-scale operates on local cost functions' gradients. We show that
with a proper choices for the step-sizes' parameters, the algorithm achieves a
convergence rate of $\mathcal{O}({T}^{-1/3 + \epsilon})$ for non-convex
distributed optimization problems over time-varying networks, for any
$\epsilon>0$. Our simulation results support the theoretical results of the
paper.
