  We propose an improved version of the SMO algorithm for training
classification and regression SVMs, based on a Conjugate Descent procedure.
This new approach only involves a modest increase on the computational cost of
each iteration but, in turn, usually results in a substantial decrease in the
number of iterations required to converge to a given precision. Besides, we
prove convergence of the iterates of this new Conjugate SMO as well as a linear
rate when the kernel matrix is positive definite. We have implemented Conjugate
SMO within the LIBSVM library and show experimentally that it is faster for
many hyper-parameter configurations, being often a better option than second
order SMO when performing a grid-search for SVM tuning.
