  This paper considers the problem of kernel regression and classification with
possibly unobservable response variables in the data, where the mechanism that
causes the absence of information is unknown and can depend on both predictors
and the response variables. Our proposed approach involves two steps: In the
first step, we construct a family of models (possibly infinite dimensional)
indexed by the unknown parameter of the missing probability mechanism. In the
second step, a search is carried out to find the empirically optimal member of
an appropriate cover (or subclass) of the underlying family in the sense of
minimizing the mean squared prediction error. The main focus of the paper is to
look into the theoretical properties of these estimators. The issue of
identifiability is also addressed. Our methods use a data-splitting approach
which is quite easy to implement. We also derive exponential bounds on the
performance of the resulting estimators in terms of their deviations from the
true regression curve in general Lp norms, where we also allow the size of the
cover or subclass to diverge as the sample size n increases. These bounds
immediately yield various strong convergence results for the proposed
estimators. As an application of our findings, we consider the problem of
statistical classification based on the proposed regression estimators and also
look into their rates of convergence under different settings. Although this
work is mainly stated for kernel-type estimators, they can also be extended to
other popular local-averaging methods such as nearest-neighbor estimators, and
histogram estimators.
