  The stochastic gradient descent (SGD) algorithm is widely used for parameter
estimation, especially for huge data sets and online learning. While this
recursive algorithm is popular for computation and memory efficiency,
quantifying variability and randomness of the solutions has been rarely
studied. This paper aims at conducting statistical inference of SGD-based
estimates in an online setting. In particular, we propose a fully online
estimator for the covariance matrix of averaged SGD iterates (ASGD) only using
the iterates from SGD. We formally establish our online estimator's consistency
and show that the convergence rate is comparable to offline counterparts. Based
on the classic asymptotic normality results of ASGD, we construct
asymptotically valid confidence intervals for model parameters. Upon receiving
new observations, we can quickly update the covariance matrix estimate and the
confidence intervals. This approach fits in an online setting and takes full
advantage of SGD: efficiency in computation and memory.
