  Automatic extraction of clinical concepts is an essential step for turning
the unstructured data within a clinical note into structured and actionable
information. In this work, we propose a clinical concept extraction model for
automatic annotation of clinical problems, treatments, and tests in clinical
notes utilizing domain-specific contextual word embedding. A contextual word
embedding model is first trained on a corpus with a mixture of clinical reports
and relevant Wikipedia pages in the clinical domain. Next, a bidirectional
LSTM-CRF model is trained for clinical concept extraction using the contextual
word embedding model. We tested our proposed model on the I2B2 2010 challenge
dataset. Our proposed model achieved the best performance among reported
baseline models and outperformed the state-of-the-art models by 3.4% in terms
of F1-score.
