  We hypothesize that large language models (LLMs) based on the transformer
architecture can enable automated detection of clinical phenotype terms,
including terms not documented in the HPO. In this study, we developed two
types of models: PhenoBCBERT, a BERT-based model, utilizing Bio+Clinical BERT
as its pre-trained model, and PhenoGPT, a GPT-based model that can be
initialized from diverse GPT models, including open-source versions such as
GPT-J, Falcon, and LLaMA, as well as closed-source versions such as GPT-3 and
GPT-3.5. We compared our methods with PhenoTagger, a recently developed HPO
recognition tool that combines rule-based and deep learning methods. We found
that our methods can extract more phenotype concepts, including novel ones not
characterized by HPO. We also performed case studies on biomedical literature
to illustrate how new phenotype information can be recognized and extracted. We
compared current BERT-based versus GPT-based models for phenotype tagging, in
multiple aspects including model architecture, memory usage, speed, accuracy,
and privacy protection. We also discussed the addition of a negation step and
an HPO normalization layer to the transformer models for improved HPO term
tagging. In conclusion, PhenoBCBERT and PhenoGPT enable the automated discovery
of phenotype terms from clinical notes and biomedical literature, facilitating
automated downstream tasks to derive new biological insights on human diseases.
