  Optimization lies at the heart of machine learning and signal processing.
Contemporary approaches based on the stochastic gradient method are
non-adaptive in the sense that their implementation employs prescribed
parameter values that need to be tuned for each application. This article
summarizes recent research and motivates future work on adaptive stochastic
optimization methods, which have the potential to offer significant
computational savings when training large-scale systems.
