  Prior to clinical applications, it is critical that risk prediction models
are evaluated in independent studies that did not contribute to model
development. While prospective cohort studies provide a natural setting for
model validation, they often ascertain information on some risk factors (e.g.,
an expensive biomarker) in a nested sub-study of the original cohort, typically
selected based on case-control status, and possibly some additional covariates.
In this article, we propose an efficient approach for evaluating discriminatory
ability of models using data from all individuals in a cohort study
irrespective of whether they were sampled in the nested sub-study for measuring
the complete set of risk factors. For evaluation of the Area Under the Curve
(AUC) statistics, we estimate probabilities of risk-scores for cases being
larger than those in controls conditional on partial risk-scores, the component
of the risk-score that could be defined based on partial covariate information.
The use of partial risk-scores, as opposed to actual multivariate risk-factor
profiles, allows estimation of the underlying conditional expectations using
subjects with complete covariate information in a non-parametric fashion even
when numerous covariates are involved. We propose an influence function based
approach for estimation of the variance of the resulting AUC statistics. We
evaluate finite sample performance of the proposed method and compare it to an
inverse probability weighted (IPW) estimator through extensive simulation
studies. Finally, we illustrate an application of the proposed method for
evaluating performance of a lung cancer risk prediction model using data from
the Prostate, Lung, Colorectal and Ovarian Cancer (PLCO) trial.
