  Clinical trials are pivotal in medical research, and NLP can enhance their
success, with application in recruitment. This study aims to evaluate the
generalizability of eligibility classification across a broad spectrum of
clinical trials. Starting with phase 3 cancer trials, annotated with seven
eligibility exclusions, then to determine how well models can generalize to
non-cancer and non-phase 3 trials. To assess this, we have compiled eligibility
criteria data for five types of trials: (1) additional phase 3 cancer trials,
(2) phase 1 and 2 cancer trials, (3) heart disease trials, (4) type 2 diabetes
trials, and (5) observational trials for any disease, comprising 2,490
annotated eligibility criteria across seven exclusion types. Our results show
that models trained on the extensive cancer dataset can effectively handle
criteria commonly found in non-cancer trials, such as autoimmune diseases.
However, they struggle with criteria disproportionately prevalent in cancer
trials, like prior malignancy. We also experiment with few-shot learning,
demonstrating that a limited number of disease-specific examples can partially
overcome this performance gap. We are releasing this new dataset of annotated
eligibility statements to promote the development of cross-disease
generalization in clinical trial classification.
