  This paper focuses on coordinate update methods, which are useful for solving
problems involving large or high-dimensional datasets. They decompose a problem
into simple subproblems, where each updates one, or a small block of, variables
while fixing others. These methods can deal with linear and nonlinear mappings,
smooth and nonsmooth functions, as well as convex and nonconvex problems. In
addition, they are easy to parallelize.
  The great performance of coordinate update methods depends on solving simple
sub-problems. To derive simple subproblems for several new classes of
applications, this paper systematically studies coordinate-friendly operators
that perform low-cost coordinate updates.
  Based on the discovered coordinate friendly operators, as well as operator
splitting techniques, we obtain new coordinate update algorithms for a variety
of problems in machine learning, image processing, as well as sub-areas of
optimization. Several problems are treated with coordinate update for the first
time in history. The obtained algorithms are scalable to large instances
through parallel and even asynchronous computing. We present numerical examples
to illustrate how effective these algorithms are.
