  Many commonly used statistical estimators are derived from optimization
problems. This includes maximum likelihood estimation, empirical risk
minimization, and so on. In many cases, the resulting estimators can be written
as solutions to estimating equations, sometimes referred to as $Z$-estimators.
Asymptotic normality for $Z$-estimators is a well-known result albeit when the
dimension of the parameter is asymptotically smaller than the square root of
the sample size. This hinders statistical inference when the dimension is
"large." In this paper, we propose a self-normalization-based confidence set
bypassing the asymptotic normality results. The proposed method is valid in the
full range of dimensions growing smaller than the sample size (ignoring
logarithmic factors) and asymptotically matches the asymptotic normality based
confidence sets when asymptotic normality holds. Our proposal represents the
first such general construction of confidence sets in the full range of
consistency of $Z$-estimators.
