  Background: Biomedical entity normalization is critical to biomedical
research because the richness of free-text clinical data, such as progress
notes, can often be fully leveraged only after translating words and phrases
into structured and coded representations suitable for analysis. Large Language
Models (LLMs), in turn, have shown great potential and high performance in a
variety of natural language processing (NLP) tasks, but their application for
normalization remains understudied.
  Methods: We applied both proprietary and open-source LLMs in combination with
several rule-based normalization systems commonly used in biomedical research.
We used a two-step LLM integration approach, (1) using an LLM to generate
alternative phrasings of a source utterance, and (2) to prune candidate UMLS
concepts, using a variety of prompting methods. We measure results by
$F_{\beta}$, where we favor recall over precision, and F1.
  Results: We evaluated a total of 5,523 concept terms and text contexts from a
publicly available dataset of human-annotated biomedical abstracts.
Incorporating GPT-3.5-turbo increased overall $F_{\beta}$ and F1 in
normalization systems +9.5 and +7.3 (MetaMapLite), +13.9 and +10.9 (QuickUMLS),
and +10.5 and +10.3 (BM25), while the open-source Vicuna model achieved +10.8
and +12.2 (MetaMapLite), +14.7 and +15 (QuickUMLS), and +15.6 and +18.7 (BM25).
  Conclusions: Existing general-purpose LLMs, both propriety and open-source,
can be leveraged at scale to greatly improve normalization performance using
existing tools, with no fine-tuning.
