  Stochastic gradient descent (SGD) is a standard optimization method to
minimize a training error with respect to network parameters in modern neural
network learning. However, it typically suffers from proliferation of saddle
points in the high-dimensional parameter space. Therefore, it is highly
desirable to design an efficient algorithm to escape from these saddle points
and reach a parameter region of better generalization capabilities. Here, we
propose a simple extension of SGD, namely reinforced SGD, which simply adds
previous first-order gradients in a stochastic manner with a probability that
increases with learning time. As verified in a simple synthetic dataset, this
method significantly accelerates learning compared with the original SGD.
Surprisingly, it dramatically reduces over-fitting effects, even compared with
state-of-the-art adaptive learning algorithm---Adam. For a benchmark
handwritten digits dataset, the learning performance is comparable to Adam, yet
with an extra advantage of requiring one-fold less computer memory. The
reinforced SGD is also compared with SGD with fixed or adaptive momentum
parameter and Nesterov's momentum, which shows that the proposed framework is
able to reach a similar generalization accuracy with less computational costs.
Overall, our method introduces stochastic memory into gradients, which plays an
important role in understanding how gradient-based training algorithms can work
and its relationship with generalization abilities of deep networks.
