  We propose an online algorithm for tracking a multidimensional time-varying
parameter of a time series, which is also allowed to be a predictable process
with respect to the underlying time series. The algorithm is driven by a gain
function. Under assumptions on the gain, we derive uniform non-asymptotic error
bounds on the tracking algorithm in terms of chosen step size for the algorithm
and the variation of the parameter of interest. We also outline how appropriate
gain functions can be constructed. We give several examples of different
variational setups for the parameter process where our result can be applied.
The proposed approach covers many frameworks and models (including the
classical Robbins-Monro and Kiefer-Wolfowitz procedures) where stochastic
approximation algorithms comprise the main inference tool for the data
analysis. We treat in some detail a couple of specific models.
