  Statistical methods with empirical likelihood (EL) are appealing and
effective especially in conjunction with estimating equations through which
useful data information can be adaptively and flexibly incorporated. It is also
known in the literature that EL approaches encounter difficulties when dealing
with problems having high-dimensional model parameters and estimating
equations. To overcome the challenges, we begin our study with a careful
investigation on high-dimensional EL from a new scope targeting at estimating a
high-dimensional sparse model parameters. We show that the new scope provides
an opportunity for relaxing the stringent requirement on the dimensionality of
the model parameter. Motivated by the new scope, we then propose a new
penalized EL by applying two penalty functions respectively regularizing the
model parameters and the associated Lagrange multipliers in the optimizations
of EL. By penalizing the Lagrange multiplier to encourage its sparsity, we show
that drastic dimension reduction in the number of estimating equations can be
effectively achieved without compromising the validity and consistency of the
resulting estimators. Most attractively, such a reduction in dimensionality of
estimating equations is actually equivalent to a selection among those
high-dimensional estimating equations, resulting in a highly parsimonious and
effective device for high-dimensional sparse model parameters. Allowing both
the dimensionalities of model parameters and estimating equations growing
exponentially with the sample size, our theory demonstrates that the estimator
from our new penalized EL is sparse and consistent with asymptotically normally
distributed nonzero components. Numerical simulations and a real data analysis
show that the proposed penalized EL works promisingly.
