  This paper provides an alternative to penalized estimators for estimation and
vari- able selection in high dimensional linear regression models with
measurement error or missing covariates. We propose estimation via bias
corrected least squares after model selection. We show that by separating model
selection and estimation, it is possible to achieve an improved rate of
convergence of the L2 estimation error compared to the rate sqrt{s log p/n}
achieved by simultaneous estimation and variable selection methods such as L1
penalized corrected least squares. If the correct model is selected with high
probability then the L2 rate of convergence for the proposed method is indeed
the oracle rate of sqrt{s/n}. Here s, p are the number of non zero parameters
and the model dimension, respectively, and n is the sample size. Under very
general model selection criteria, the proposed method is computationally
simpler and statistically at least as efficient as the L1 penalized corrected
least squares method, performs model selection without the availability of the
bias correction matrix, and is able to provide estimates with only a small
sub-block of the bias correction covariance matrix of order s x s in comparison
to the p x p correction matrix required for computation of the L1 penalized
version. Furthermore we show that the model selection requirements are met by a
correlation screening type method and the L1 penalized corrected least squares
method. Also, the proposed methodology when applied to the estimation of
precision matrices with missing observations, is seen to perform at least as
well as existing L1 penalty based methods. All results are supported
empirically by a simulation study.
