  The primal-dual method of multipliers (PDMM) was originally designed for
solving a decomposable optimisation problem over a general network. In this
paper, we revisit PDMM for optimisation over a centralized network. We first
note that the recently proposed method FedSplit [1] implements PDMM for a
centralized network. In [1], Inexact FedSplit (i.e., gradient based FedSplit)
was also studied both empirically and theoretically. We identify the cause for
the poor reported performance of Inexact FedSplit, which is due to the improper
initialisation in the gradient operations at the client side. To fix the issue
of Inexact FedSplit, we propose two versions of Inexact PDMM, which are
referred to as gradient-based PDMM (GPDMM) and accelerated GPDMM (AGPDMM),
respectively. AGPDMM accelerates GPDMM at the cost of transmitting two times
the number of parameters from the server to each client per iteration compared
to GPDMM. We provide a new convergence bound for GPDMM for a class of convex
optimisation problems. Our new bounds are tighter than those derived for
Inexact FedSplit. We also investigate the update expressions of AGPDMM and
SCAFFOLD to find their similarities. It is found that when the number K of
gradient steps at the client side per iteration is K=1, both AGPDMM and
SCAFFOLD reduce to vanilla gradient descent with proper parameter setup.
Experimental results indicate that AGPDMM converges faster than SCAFFOLD when
K>1 while GPDMM converges slightly worse than SCAFFOLD.
