  Multivariable predictive models are important statistical tools for providing
synthetic diagnosis and prognostic algorithms based on multiple patients'
characteristics. Their apparent discriminant and calibration measures usually
have overestimation biases (known as 'optimism') relative to the actual
performances for external populations. Existing statistical evidence and
guidelines suggest that three bootstrap-based bias correction methods are
preferable in practice, namely Harrell's bias correction and the .632 and .632+
estimators. Although Harrell's method has been widely adopted in clinical
studies, simulation-based evidence indicates that the .632+ estimator may
perform better than the other two methods. However, there is limited evidence
and these methods' actual comparative effectiveness is still unclear. In this
article, we conducted extensive simulations to compare the effectiveness of
these methods, particularly using the following modern regression models:
conventional logistic regression, stepwise variable selections, Firth's
penalized likelihood method, ridge, lasso, and elastic-net. Under relatively
large sample settings, the three bootstrap-based methods were comparable and
performed well. However, all three methods had biases under small sample
settings, and the directions and sizes of the biases were inconsistent. In
general, the .632+ estimator is recommended, but we provide several notes
concerning the operating characteristics of each method.
