  The objective of our work is to demonstrate the feasibility of utilizing deep
learning models to extract safety signals related to the use of dietary
supplements (DS) in clinical text. Two tasks were performed in this study. For
the named entity recognition (NER) task, Bi-LSTM-CRF (Bidirectional
Long-Short-Term-Memory Conditional Random Fields) and BERT (Bidirectional
Encoder Representations from Transformers) models were trained and compared
with CRF model as a baseline to recognize the named entities of DS and Events
from clinical notes. In the relation extraction (RE) task, two deep learning
models, including attention-based Bi-LSTM and CNN (Convolutional Neural
Network), and a random forest model were trained to extract the relations
between DS and Events, which were categorized into three classes: positive
(i.e., indication), negative (i.e., adverse events), and not related. The best
performed NER and RE models were further applied on clinical notes mentioning
88 DS for discovering DS adverse events and indications, which were compared
with a DS knowledge base. For the NER task, deep learning models achieved a
better performance than CRF, with F1 scores above 0.860. The attention-based
Bi-LSTM model performed the best in the relation extraction task, with the F1
score of 0.893. When comparing DS event pairs generated by the deep learning
models with the knowledge base for DS and Event, we found both known and
unknown pairs. Deep learning models can detect adverse events and indication of
DS in clinical notes, which hold great potential for monitoring the safety of
DS use.
