  In this paper, we present a unified and general framework for analyzing the
batch updating approach to nonlinear, high-dimensional optimization. The
framework encompasses all the currently used batch updating approaches, and is
applicable to nonconvex as well as convex functions. Moreover, the framework
permits the use of noise-corrupted gradients, as well as first-order
approximations to the gradient (sometimes referred to as "gradient-free"
approaches). By viewing the analysis of the iterations as a problem in the
convergence of stochastic processes, we are able to establish a very general
theorem, which includes most known convergence results for zeroth-order and
first-order methods. The analysis of "second-order" or momentum-based methods
is not a part of this paper, and will be studied elsewhere. However, numerical
experiments indicate that momentum-based methods can fail if the true gradient
is replaced by its first-order approximation. This requires further theoretical
analysis.
