  In this paper, we concentrate on decentralized optimization problems with
nonconvex and nonsmooth objective functions, especially on the decentralized
training of nonsmooth neural networks. We introduce a unified framework to
analyze the global convergence of decentralized stochastic subgradient-based
methods. We prove the global convergence of our proposed framework under mild
conditions, by establishing that the generated sequence asymptotically
approximates the trajectories of its associated differential inclusion.
Furthermore, we establish that our proposed framework covers a wide range of
existing efficient decentralized subgradient-based methods, including
decentralized stochastic subgradient descent (DSGD), DSGD with
gradient-tracking technique (DSGD-T), and DSGD with momentum (DSGD-M). In
addition, we introduce the sign map to regularize the update directions in
DSGD-M, and show it is enclosed in our proposed framework. Consequently, our
convergence results establish, for the first time, global convergence of these
methods when applied to nonsmooth nonconvex objectives. Preliminary numerical
experiments demonstrate that our proposed framework yields highly efficient
decentralized subgradient-based methods with convergence guarantees in the
training of nonsmooth neural networks.
