  A novel approach is given to overcome the computational challenges of the
full-matrix Adaptive Gradient algorithm (Full AdaGrad) in stochastic
optimization. By developing a recursive method that estimates the inverse of
the square root of the covariance of the gradient, alongside a streaming
variant for parameter updates, the study offers efficient and practical
algorithms for large-scale applications. This innovative strategy significantly
reduces the complexity and resource demands typically associated with
full-matrix methods, enabling more effective optimization processes. Moreover,
the convergence rates of the proposed estimators and their asymptotic
efficiency are given. Their effectiveness is demonstrated through numerical
studies.
