  Cross-validation is frequently used for model selection in a variety of
applications. However, it is difficult to apply cross-validation to mixed
effects models (including nonlinear mixed effects models or NLME models) due to
the fact that cross-validation requires "out-of-sample" predictions of the
outcome variable, which cannot be easily calculated when random effects are
present. We describe two novel variants of cross-validation that can be applied
to nonlinear mixed effects models. One variant, where out-of-sample predictions
are based on post hoc estimates of the random effects, can be used to select
the overall structural model. Another variant, where cross-validation seeks to
minimize the estimated random effects rather than the estimated residuals, can
be used to select covariates to include in the model. We show that these
methods produce accurate results in a variety of simulated data sets and apply
them to two publicly available population pharmacokinetic data sets.
