  Computing the gradient of a function provides fundamental information about
its behavior. This information is essential for several applications and
algorithms across various fields. One common application that require gradients
are optimization techniques such as stochastic gradient descent, Newton's
method and trust region methods. However, these methods usually requires a
numerical computation of the gradient at every iteration of the method which is
prone to numerical errors. We propose a simple limited-memory technique for
improving the accuracy of a numerically computed gradient in this
gradient-based optimization framework by exploiting (1) a coordinate
transformation of the gradient and (2) the history of previously taken descent
directions. The method is verified empirically by extensive experimentation on
both test functions and on real data applications. The proposed method is
implemented in the R package smartGrad and in C++.
