  The purpose of this paper is to improve upon existing variants of gradient
descent by solving two problems: (1) removing (or reducing) the plateau that
occurs while minimizing the cost function, (2) continually adjusting the
learning rate to an "ideal" value. The approach taken is to approximately solve
for the learning rate as a function of a trust metric. When this technique is
hybridized with momentum, it creates an especially effective gradient descent
variant, called NeogradM. It is shown to outperform Adam on several test
problems, and can easily reach cost function values that are smaller by a
factor of $10^8$, for example.
