  Large pre-trained language models (LLMs) have been shown to have significant
potential in few-shot learning across various fields, even with minimal
training data. However, their ability to generalize to unseen tasks in more
complex fields, such as biology, has yet to be fully evaluated. LLMs can offer
a promising alternative approach for biological inference, particularly in
cases where structured data and sample size are limited, by extracting prior
knowledge from text corpora. Our proposed few-shot learning approach uses LLMs
to predict the synergy of drug pairs in rare tissues that lack structured data
and features. Our experiments, which involved seven rare tissues from different
cancer types, demonstrated that the LLM-based prediction model achieved
significant accuracy with very few or zero samples. Our proposed model, the
CancerGPT (with $\sim$ 124M parameters), was even comparable to the larger
fine-tuned GPT-3 model (with $\sim$ 175B parameters). Our research is the first
to tackle drug pair synergy prediction in rare tissues with limited data. We
are also the first to utilize an LLM-based prediction model for biological
reaction prediction tasks.
