  For a symmetric positive semidefinite linear system of equations $\mathcal{Q}
{\bf x} = {\bf b}$, where ${\bf x} = (x_1,\ldots,x_s)$ is partitioned into $s$
blocks, with $s \geq 2$, we show that each cycle of the classical block
symmetric Gauss-Seidel (block sGS) method exactly solves the associated
quadratic programming (QP) problem but added with an extra proximal term of the
form $\frac{1}{2} \| {\bf x}-{\bf x}^k \|_{\mathcal T}^2$, where ${\mathcal T}$
is a symmetric positive semidefinite matrix related to the sGS decomposition
and ${\bf x}^k$ is the previous iterate. By leveraging on such a connection to
optimization, we are able to extend the result (which we name as the block sGS
decomposition theorem) for solving a convex composite QP (CCQP) with an
additional possibly nonsmooth term in $x_1$, i.e., $\min\{ p(x_1) +
\frac{1}{2}\langle {\bf x},\, \mathcal{Q} {\bf x} \rangle -\langle {\bf b},\,
{\bf x}\rangle\}$, where $p(\cdot)$ is a proper closed convex function. Based
on the block sGS decomposition theorem, we are able to extend the classical
block sGS method to solve a CCQP. In addition, our extended block sGS method
has the flexibility of allowing for inexact computation in each step of the
block sGS cycle. At the same time, we can also accelerate the inexact block sGS
method to achieve an iteration complexity of $O(1/k^2)$ after performing $k$
block sGS cycles. As a {fundamental} building block, the block sGS
decomposition theorem has played a key role in various recently developed
algorithms such as the inexact semiproximal {ALM/ADMM} for linearly constrained
multi-block convex composite conic programming (CCCP), and the accelerated
block coordinate descent method for multi-block CCCP.
