  Clinical notes contain information about patients that goes beyond structured
data like lab values and medications. However, clinical notes have been
underused relative to structured data, because notes are high-dimensional and
sparse. This work develops and evaluates representations of clinical notes
using bidirectional transformers (ClinicalBERT). ClinicalBERT uncovers
high-quality relationships between medical concepts as judged by humans.
ClinicalBert outperforms baselines on 30-day hospital readmission prediction
using both discharge summaries and the first few days of notes in the intensive
care unit. Code and model parameters are available.
