  The existing machine learning algorithms for minimizing the convex function
over a closed convex set suffer from slow convergence because their learning
rates must be determined before running them. This paper proposes two machine
learning algorithms incorporating the line search method, which automatically
and algorithmically finds appropriate learning rates at run-time. One algorithm
is based on the incremental subgradient algorithm, which sequentially and
cyclically uses each of the parts of the objective function; the other is based
on the parallel subgradient algorithm, which uses parts independently in
parallel. These algorithms can be applied to constrained nonsmooth convex
optimization problems appearing in tasks of learning support vector machines
without adjusting the learning rates precisely. The proposed line search method
can determine learning rates to satisfy weaker conditions than the ones used in
the existing machine learning algorithms. This implies that the two algorithms
are generalizations of the existing incremental and parallel subgradient
algorithms for solving constrained nonsmooth convex optimization problems. We
show that they generate sequences that converge to a solution of the
constrained nonsmooth convex optimization problem under certain conditions. The
main contribution of this paper is the provision of three kinds of experiment
showing that the two algorithms can solve concrete experimental problems faster
than the existing algorithms. First, we show that the proposed algorithms have
performance advantages over the existing ones in solving a test problem.
Second, we compare the proposed algorithms with a different algorithm Pegasos,
which is designed to learn with a support vector machine efficiently, in terms
of prediction accuracy, value of the objective function, and computational
time. Finally, we use...
