  The current mode of use of Electronic Health Record (EHR) elicits text
redundancy. Clinicians often populate new documents by duplicating existing
notes, then updating accordingly. Data duplication can lead to a propagation of
errors, inconsistencies and misreporting of care. Therefore, quantifying
information redundancy can play an essential role in evaluating innovations
that operate on clinical narratives.
  This work is a quantitative examination of information redundancy in EHR
notes. We present and evaluate two strategies to measure redundancy: an
information-theoretic approach and a lexicosyntactic and semantic model. We
evaluate the measures by training large Transformer-based language models using
clinical text from a large openly available US-based ICU dataset and a large
multi-site UK based Trust. By comparing the information-theoretic content of
the trained models with open-domain language models, the language models
trained using clinical text have shown ~1.5x to ~3x less efficient than
open-domain corpora. Manual evaluation shows a high correlation with
lexicosyntactic and semantic redundancy, with averages ~43 to ~65%.
