  Deep learning models have shown tremendous potential in learning
representations, which are able to capture some key properties of the data.
This makes them great candidates for transfer learning: Exploiting
commonalities between different learning tasks to transfer knowledge from one
task to another. Electronic health records (EHR) research is one of the domains
that has witnessed a growing number of deep learning techniques employed for
learning clinically-meaningful representations of medical concepts (such as
diseases and medications). Despite this growth, the approaches to benchmark and
assess such learned representations (or, embeddings) is under-investigated;
this can be a big issue when such embeddings are shared to facilitate transfer
learning. In this study, we aim to (1) train some of the most prominent disease
embedding techniques on a comprehensive EHR data from 3.1 million patients, (2)
employ qualitative and quantitative evaluation techniques to assess these
embeddings, and (3) provide pre-trained disease embeddings for transfer
learning. This study can be the first comprehensive approach for clinical
concept embedding evaluation and can be applied to any embedding techniques and
for any EHR concept.
