  The so-called fast inertial relaxation engine is a first-order method for
unconstrained smooth optimization problems. It updates the search direction by
a linear combination of the past search direction, the current gradient and the
normalized gradient direction. We explore more general combination rules and
call this generalized technique as the search direction correction (SDC). SDC
is extended to composite and stochastic optimization problems as well. Deriving
from a second-order ODE, we propose a fast inertial search direction correction
(FISC) algorithm as an example of methods with SDC. We prove the
$\mathcal{O}(k^{-2})$ convergence rate of FISC for convex optimization
problems. Numerical results on sparse optimization, logistic regression as well
as deep learning demonstrate that our proposed methods are quite competitive to
other state-of-the-art first-order algorithms.
