  Gradient methods are experiencing a growth in methodological and theoretical
developments owing to the challenges posed by optimization problems arising in
data science. However, such gradient methods face diverging optimality gaps or
exploding objective evaluations when applied to optimization problems with
realistic properties for data science applications. In this work, we address
this gap by developing a generic methodology that economically uses objective
function evaluations in a problem-driven manner to prevent optimality gap
divergence and avoid explosions in objective evaluations. Our methodology
allows for a variety of step size routines and search direction strategies.
Furthermore, we develop a particular, novel step size selection methodology
that is well-suited to our framework. We show that our specific procedure is
highly competitive with standard optimization methods on CUTEst test problems.
We then show our specific procedure is highly favorable relative to standard
optimization methods on a particularly tough data science problem: learning the
parameters in a generalized estimating equation model. Thus, we provide a novel
gradient methodology that is better suited to optimization problems from this
important class of data science applications.
