  The alternating direction method of multipliers (ADMM) has been popular for
solving many signal processing problems, convex or nonconvex. In this paper, we
study an asynchronous implementation of the ADMM for solving a nonconvex
nonsmooth optimization problem, whose objective is the sum of a number of
component functions. The proposed algorithm allows the problem to be solved in
a distributed, asynchronous and incremental manner. First, the component
functions can be distributed to different computing nodes, who perform the
updates asynchronously without coordinating with each other. Two sources of
asynchrony are covered by our algorithm: one is caused by the heterogeneity of
the computational nodes, and the other arises from unreliable communication
links. Second, the algorithm can be viewed as implementing an incremental
algorithm where at each step the (possibly delayed) gradients of only a subset
of component functions are update d. We show that when certain bounds are put
on the level of asynchrony, the proposed algorithm converges to the set of
stationary solutions (resp. optimal solutions) for the nonconvex (resp. convex)
problem. To the best of our knowledge, the proposed ADMM implementation can
tolerate the highest degree of asynchrony, among all known asynchronous
variants of the ADMM. Moreover, it is the first ADMM implementation that can
deal with nonconvexity and asynchrony at the same time.
