  High-throughput phenotyping automates the mapping of patient signs to
standardized ontology concepts and is essential for precision medicine. This
study evaluates the automation of phenotyping of clinical summaries from the
Online Mendelian Inheritance in Man (OMIM) database using large language
models. Due to their rich phenotype data, these summaries can be surrogates for
physician notes. We conduct a performance comparison of GPT-4 and
GPT-3.5-Turbo. Our results indicate that GPT-4 surpasses GPT-3.5-Turbo in
identifying, categorizing, and normalizing signs, achieving concordance with
manual annotators comparable to inter-rater agreement. Despite some limitations
in sign normalization, the extensive pre-training of GPT-4 results in high
performance and generalizability across several phenotyping tasks while
obviating the need for manually annotated training data. Large language models
are expected to be the dominant method for automating high-throughput
phenotyping of clinical text.
