  Dealing with missing data is an important problem in statistical analysis
that is often addressed with imputation procedures. The performance and
validity of such methods are of great importance for their application in
empirical studies. While the prevailing method of Multiple Imputation by
Chained Equations (MICE) with Predictive Mean Matching (PMM) is considered
standard in the social science literature, the increase in complex datasets may
require more advanced approaches based on machine learning. In particular,
tree-based imputation methods have emerged as very competitive approaches.
However, the performance and validity are not completely understood,
particularly compared to the standard MICE PMM. This is especially true for
inference in linear models. In this study, we investigate the impact of various
imputation methods on coefficient estimation, Type I error, and power, to gain
insights that can help empirical researchers deal with missingness more
effectively. We explore MICE PMM alongside different tree-based methods, such
as MICE with Random Forest (RF), Chained Random Forests with and without PMM
(missRanger), and Extreme Gradient Boosting (MIXGBoost), conducting a realistic
simulation study using the German National Educational Panel Study (NEPS) as
the original data source. Our results reveal that Random Forest-based
imputations, especially MICE RF and missRanger with PMM, consistently perform
better in most scenarios. Standard MICE PMM shows partially increased bias and
overly conservative test decisions, particularly with non-true zero
coefficients. Our results thus underscore the potential advantages of
tree-based imputation methods, albeit with a caveat that all methods perform
worse with an increased missingness, particularly missRanger.
