  We study the problem of determining the optimal exploration strategy in an
unconstrained scalar optimization problem depending on an unknown parameter to
be learned from online collected noisy data. An optimal trade-off between
exploration and exploitation is crucial for effective optimization under
uncertainties, and to achieve this we consider a cumulative regret minimization
approach over a finite horizon, with each time instant in the horizon
characterized by a stochastic exploration signal, whose variance has to be
designed. In this setting, under an idealized assumption on an appropriately
defined information function associated with the excitation, we are able to
show that the optimal exploration strategy is either to use no exploration at
all (called lazy exploration) or adding an exploration excitation only at the
first time instant of the horizon (called immediate exploration). A quadratic
numerical example is used to illustrate the results.
