  Missing values have been thoroughly analyzed in the context of linear models,
where the final aim is to build coefficient estimates. However, estimating
coefficients does not directly solve the problem of prediction with missing
entries: a manner to address empty components must be designed. Major
approaches to deal with prediction with missing values are empirically driven
and can be decomposed into two families: imputation (filling in empty fields)
and pattern-by-pattern prediction, where a predictor is built on each missing
pattern. Unfortunately, most simple imputation techniques used in practice (as
constant imputation) are not consistent when combined with linear models. In
this paper, we focus on the more flexible pattern-by-pattern approaches and
study their predictive performances on Missing Completely At Random (MCAR)
data. We first show that a pattern-by-pattern logistic regression model is
intrinsically ill-defined, implying that even classical logistic regression is
impossible to apply to missing data. We then analyze the perceptron model and
show how the linear separability property extends to partially-observed inputs.
Finally, we use the Linear Discriminant Analysis to prove that
pattern-by-pattern LDA is consistent in a high-dimensional regime. We refine
our analysis to more complex MNAR data.
