  In this paper, a centralized two-block separable optimization is considered
for which a fully parallel primal-dual discrete-time algorithm with fixed step
size is derived based on monotone operator splitting method. In this algorithm,
the primal variables are updated in an alternating fashion like Alternating
Direction Method of Multipliers (ADMM). However, unlike existing discrete-time
algorithms such as Method of Multipliers (MM), ADMM, Bi-Alternating Direction
Method of Multipliers (BiADMM), and Primal-Dual Fixed Point (PDFP) algorithms,
that all suffer from sequential updates, all primal and dual variables are
updated in parallel in the sense that to update a variable at each time,
updated version of other variable(s) is not required. One of advantages of the
proposed algorithm is that its direct extension to multi-block optimization is
still convergent. Then the method is applied to distributed optimization for
which a fully parallel primal-dual distributed algorithm is obtained. Finally,
since direct extension of ADMM may diverge for multi-block optimization, a
numerical example of a three-block optimization is given for which the direct
extension of the proposed algorithm is shown to converge to a solution.
