  Using an optimization algorithm to solve a machine learning problem is one of
mainstreams in the field of science. In this work, we demonstrate a
comprehensive comparison of some state-of-the-art first-order optimization
algorithms for convex optimization problems in machine learning. We concentrate
on several smooth and non-smooth machine learning problems with a loss function
plus a regularizer. The overall experimental results show the superiority of
primal-dual algorithms in solving a machine learning problem from the
perspectives of the ease to construct, running time and accuracy.
