  Recently, there is great interest to investigate the application of deep
learning models for the prediction of clinical events using electronic health
records (EHR) data. In EHR data, a patient's history is often represented as a
sequence of visits, and each visit contains multiple events. As a result, deep
learning models developed for sequence modeling, like recurrent neural networks
(RNNs) are common architecture for EHR-based clinical events predictive models.
While a large variety of RNN models were proposed in the literature, it is
unclear if complex architecture innovations will offer superior predictive
performance. In order to move this field forward, a rigorous evaluation of
various methods is needed. In this study, we conducted a thorough benchmark of
RNN architectures in modeling EHR data. We used two prediction tasks: the risk
for developing heart failure and the risk of early readmission for inpatient
hospitalization. We found that simple gated RNN models, including GRUs and
LSTMs, often offer competitive results when properly tuned with Bayesian
Optimization, which is in line with similar to findings in the natural language
processing (NLP) domain. For reproducibility, Our codebase is shared at
https://github.com/ZhiGroup/pytorch_ehr.
