  In the era of Large Language Models (LLMs), given their remarkable text
understanding and generation abilities, there is an unprecedented opportunity
to develop new, LLM-based methods for trustworthy medical knowledge synthesis,
extraction and summarization. This paper focuses on the problem of
Pharmacovigilance (PhV), where the significance and challenges lie in
identifying Adverse Drug Events (ADEs) from diverse text sources, such as
medical literature, clinical notes, and drug labels. Unfortunately, this task
is hindered by factors including variations in the terminologies of drugs and
outcomes, and ADE descriptions often being buried in large amounts of narrative
text. We present MALADE, the first effective collaborative multi-agent system
powered by LLM with Retrieval Augmented Generation for ADE extraction from drug
label data. This technique involves augmenting a query to an LLM with relevant
information extracted from text resources, and instructing the LLM to compose a
response consistent with the augmented data. MALADE is a general LLM-agnostic
architecture, and its unique capabilities are: (1) leveraging a variety of
external sources, such as medical literature, drug labels, and FDA tools (e.g.,
OpenFDA drug information API), (2) extracting drug-outcome association in a
structured format along with the strength of the association, and (3) providing
explanations for established associations. Instantiated with GPT-4 Turbo or
GPT-4o, and FDA drug label data, MALADE demonstrates its efficacy with an Area
Under ROC Curve of 0.90 against the OMOP Ground Truth table of ADEs. Our
implementation leverages the Langroid multi-agent LLM framework and can be
found at https://github.com/jihyechoi77/malade.
