  This paper presents a novel coordinate descent algorithm leveraging a
combination of one-directional line search and gradient information for
parameter updates for a squared error loss function. Each parameter undergoes
updates determined by either the line search or gradient method, contingent
upon whether the modulus of the gradient of the loss with respect to that
parameter surpasses a predefined threshold. Notably, a larger threshold value
enhances algorithmic efficiency. Despite the potentially slower nature of the
line search method relative to gradient descent, its parallelizability
facilitates computational time reduction. Experimental validation conducted on
a 2-layer Rectified Linear Unit network with synthetic data elucidates the
impact of hyperparameters on convergence rates and computational efficiency.
