  When cross-validating standard or extended Cox models, the commonly used
criterion is the cross-validated partial loglikelihood using a naive or a van
Houwelingen scheme -to make efficient use of the death times of the left out
data in relation to the death times of all the data-. Quite astonishingly, we
will show, using a strong simulation study involving three different data
simulation algorithms, that these two cross-validation methods fail with the
extensions, either straightforward or more involved ones, of partial least
squares regression to the Cox model. This is quite an interesting result for at
least two reasons. Firstly, several nice features of PLS based models,
including regularization, interpretability of the components, missing data
support, data visualization thanks to biplots of individuals and variables -and
even parsimony for SPLS based models-, account for a common use of these
extensions by statisticians who usually select their hyperparameters using
cross-validation. Secondly, they are almost always featured in benchmarking
studies to assess the performance of a new estimation technique used in a high
dimensional context and often show poor statistical properties. We carried out
a vast simulation study to evaluate more than a dozen of potential
cross-validation criteria, either AUC or prediction error based. Several of
them lead to the selection of a reasonable number of components. Using these
newly found cross-validation criteria to fit extensions of partial least
squares regression to the Cox model, we performed a benchmark reanalysis that
showed enhanced performances of these techniques. In addition, we defined a new
robust measure based on the Schmid score and the R coefficient of determination
for least absolute deviation: the integrated R Schmid Score weighted.
Simulations were carried out using the R-package plsRcox.
