  In this work, we introduce ADAPD, $\textbf{A}$
$\textbf{D}$ecentr$\textbf{A}$lized $\textbf{P}$rimal-$\textbf{D}$ual
algorithmic framework for solving non-convex and smooth consensus optimization
problems over a network of distributed agents. The proposed framework relies on
a novel problem formulation that elicits ADMM-type updates, where each agent
first inexactly solves a local strongly convex subproblem with any method of
its choice and then performs a neighbor communication to update a set of dual
variables. We present two variants that allow for a single gradient step for
the primal updates or multiple communications for the dual updates, to exploit
the tradeoff between the per-iteration cost and the number of iterations. When
multiple communications are performed, ADAPD can achieve theoretically optimal
communication complexity results for non-convex and smooth consensus problems.
Numerical experiments on several applications, including a deep-learning one,
demonstrate the superiority of ADAPD over several popularly used decentralized
methods.
