  Recent deep learning research based on Transformer model architectures has
demonstrated state-of-the-art performance across a variety of domains and
tasks, mostly within the computer vision and natural language processing
domains. While some recent studies have implemented Transformers for clinical
tasks using electronic health records data, they are limited in scope,
flexibility, and comprehensiveness. In this study, we propose a flexible
Transformer-based EHR embedding pipeline and predictive model framework that
introduces several novel modifications of existing workflows that capitalize on
data attributes unique to the healthcare domain. We showcase the feasibility of
our flexible design in a case study in the intensive care unit, where our
models accurately predict seven clinical outcomes pertaining to readmission and
patient mortality over multiple future time horizons.
