  The present work introduces the hybrid consensus alternating direction method
of multipliers (H-CADMM), a novel framework for optimization over networks
which unifies existing distributed optimization approaches, including the
centralized and the decentralized consensus ADMM. H-CADMM provides a flexible
tool that leverages the underlying graph topology in order to achieve a
desirable sweet-spot between node-to-node communication overhead and rate of
convergence -- thereby alleviating known limitations of both C-CADMM and
D-CADMM. A rigorous analysis of the novel method establishes linear convergence
rate, and also guides the choice of parameters to optimize this rate. The novel
hybrid update rules of H-CADMM lend themselves to "in-network acceleration"
that is shown to effect considerable -- and essentially "free-of-charge" --
performance boost over the fully decentralized ADMM. Comprehensive numerical
tests validate the analysis and showcase the potential of the method in
tackling efficiently, widely useful learning tasks.
