  Missing data is a common issue in real-world datasets. This paper studies the
performance of impute-then-regress pipelines by contrasting theoretical and
empirical evidence. We establish the asymptotic consistency of such pipelines
for a broad family of imputation methods. While common sense suggests that a
`good' imputation method produces datasets that are plausible, we show, on the
contrary, that, as far as prediction is concerned, crude can be good. Among
others, we find that mode-impute is asymptotically sub-optimal, while
mean-impute is asymptotically optimal. We then exhaustively assess the validity
of these theoretical conclusions on a large corpus of synthetic, semi-real, and
real datasets. While the empirical evidence we collect mostly supports our
theoretical findings, it also highlights gaps between theory and practice and
opportunities for future research, regarding the relevance of the MAR
assumption, the complex interdependency between the imputation and regression
tasks, and the need for realistic synthetic data generation models.
