  In longitudinal electronic health records (EHRs), the event records of a
patient are distributed over a long period of time and the temporal relations
between the events reflect sufficient domain knowledge to benefit prediction
tasks such as the rate of inpatient mortality. Medical concept embedding as a
feature extraction method that transforms a set of medical concepts with a
specific time stamp into a vector, which will be fed into a supervised learning
algorithm. The quality of the embedding significantly determines the learning
performance over the medical data. In this paper, we propose a medical concept
embedding method based on applying a self-attention mechanism to represent each
medical concept. We propose a novel attention mechanism which captures the
contextual information and temporal relationships between medical concepts. A
light-weight neural net, "Temporal Self-Attention Network (TeSAN)", is then
proposed to learn medical concept embedding based solely on the proposed
attention mechanism. To test the effectiveness of our proposed methods, we have
conducted clustering and prediction tasks on two public EHRs datasets comparing
TeSAN against five state-of-the-art embedding methods. The experimental results
demonstrate that the proposed TeSAN model is superior to all the compared
methods. To the best of our knowledge, this work is the first to exploit
temporal self-attentive relations between medical events.
