  Various deep learning algorithms have been developed to analyze different
types of clinical data including clinical text classification and extracting
information from 'free text' and so on. However, automate the keyword
extraction from the clinical notes is still challenging. The challenges include
dealing with noisy clinical notes which contain various abbreviations, possible
typos, and unstructured sentences. The objective of this research is to
investigate the attention-based deep learning models to classify the
de-identified clinical progress notes extracted from a real-world EHR system.
The attention-based deep learning models can be used to interpret the models
and understand the critical words that drive the correct or incorrect
classification of the clinical progress notes. The attention-based models in
this research are capable of presenting the human interpretable text
classification models. The results show that the fine-tuned BERT with the
attention layer can achieve a high classification accuracy of 97.6%, which is
higher than the baseline fine-tuned BERT classification model. In this
research, we also demonstrate that the attention-based models can identify
relevant keywords that are strongly related to the clinical progress note
categories.
