  Background: Existing guidelines for handling missing data are generally not
consistent with the goals of prediction modelling, where missing data can occur
at any stage of the model pipeline. Multiple imputation (MI), often heralded as
the gold standard approach, can be challenging to apply in the clinic. Clearly,
the outcome cannot be used to impute data at prediction time. Regression
imputation (RI) may offer a pragmatic alternative in the prediction context,
that is simpler to apply in the clinic. Moreover, the use of missing indicators
can handle informative missingness, but it is currently unknown how well they
perform within CPMs. Methods: We performed a simulation study where data were
generated under various missing data mechanisms to compare the predictive
performance of CPMs developed using both imputation methods. We consider
deployment scenarios where missing data is permitted/prohibited, and develop
models that use/omit the outcome during imputation and include/omit missing
indicators. Results: When complete data must be available at deployment, our
findings were in line with widely used recommendations; that the outcome should
be used to impute development data under MI, yet omitted under RI. When
imputation is applied at deployment, omitting the outcome from the imputation
at development was preferred. Missing indicators improved model performance in
some specific cases, but can be harmful when missingness is dependent on the
outcome. Conclusion: We provide evidence that commonly taught principles of
handling missing data via MI may not apply to CPMs, particularly when data can
be missing at deployment. In such settings, RI and missing indicator methods
can (marginally) outperform MI. As shown, the performance of the missing data
handling method must be evaluated on a study-by-study basis, and should be
based on whether missing data are allowed at deployment.
