  In this paper, we investigate the theoretical properties of stochastic
gradient descent (SGD) for statistical inference in the context of nonconvex
optimization problems, which have been relatively unexplored compared to convex
settings. Our study is the first to establish provable inferential procedures
using the SGD estimator for general nonconvex objective functions, which may
contain multiple local minima.
  We propose two novel online inferential procedures that combine SGD and the
multiplier bootstrap technique. The first procedure employs a consistent
covariance matrix estimator, and we establish its error convergence rate. The
second procedure approximates the limit distribution using bootstrap SGD
estimators, yielding asymptotically valid bootstrap confidence intervals. We
validate the effectiveness of both approaches through numerical experiments.
  Furthermore, our analysis yields an intermediate result: the in-expectation
error convergence rate for the original SGD estimator in nonconvex settings,
which is comparable to existing results for convex problems. We believe this
novel finding holds independent interest and enriches the literature on
optimization and statistical inference.
