  In this paper, we develop a novel second-order method for training
feed-forward neural nets. At each iteration, we construct a quadratic
approximation to the cost function in a low-dimensional subspace. We minimize
this approximation inside a trust region through a two-stage procedure: first
inside the embedded positive curvature subspace, followed by a gradient descent
step. This approach leads to a fast objective function decay, prevents
convergence to saddle points, and alleviates the need for manually tuning
parameters. We show the good performance of the proposed algorithm on benchmark
datasets.
