  With the fast development of big data, it has been easier than before to
learn the optimal decision rule by updating the decision rule recursively and
making online decisions. We study the online statistical inference of model
parameters in a contextual bandit framework of sequential decision-making. We
propose a general framework for online and adaptive data collection environment
that can update decision rules via weighted stochastic gradient descent. We
allow different weighting schemes of the stochastic gradient and establish the
asymptotic normality of the parameter estimator. Our proposed estimator
significantly improves the asymptotic efficiency over the previous averaged SGD
approach via inverse probability weights. We also conduct an optimality
analysis on the weights in a linear regression setting. We provide a Bahadur
representation of the proposed estimator and show that the remainder term in
the Bahadur representation entails a slower convergence rate compared to
classical SGD due to the adaptive data collection.
