  Radiology reports are unstructured and contain the imaging findings and
corresponding diagnoses transcribed by radiologists which include clinical
facts and negated and/or uncertain statements. Extracting pathologic findings
and diagnoses from radiology reports is important for quality control,
population health, and monitoring of disease progress. Existing works,
primarily rely either on rule-based systems or transformer-based pre-trained
model fine-tuning, but could not take the factual and uncertain information
into consideration, and therefore generate false-positive outputs. In this
work, we introduce three sedulous augmentation techniques which retain factual
and critical information while generating augmentations for contrastive
learning. We introduce RadBERT-CL, which fuses these information into BlueBert
via a self-supervised contrastive loss. Our experiments on MIMIC-CXR show
superior performance of RadBERT-CL on fine-tuning for multi-class, multi-label
report classification. We illustrate that when few labeled data are available,
RadBERT-CL outperforms conventional SOTA transformers (BERT/BlueBert) by
significantly larger margins (6-11%). We also show that the representations
learned by RadBERT-CL can capture critical medical information in the latent
space.
