  Reviewing radiology reports in emergency departments is an essential but
laborious task. Timely follow-up of patients with abnormal cases in their
radiology reports may dramatically affect the patient's outcome, especially if
they have been discharged with a different initial diagnosis. Machine learning
approaches have been devised to expedite the process and detect the cases that
demand instant follow up. However, these approaches require a large amount of
labeled data to train reliable predictive models. Preparing such a large
dataset, which needs to be manually annotated by health professionals, is
costly and time-consuming. This paper investigates a semi-supervised learning
framework for radiology report classification across three hospitals. The main
goal is to leverage clinical unlabeled data in order to augment the learning
process where limited labeled data is available. To further improve the
classification performance, we also integrate a transfer learning technique
into the semi-supervised learning pipeline . Our experimental findings show
that (1) convolutional neural networks (CNNs), while being independent of any
problem-specific feature engineering, achieve significantly higher
effectiveness compared to conventional supervised learning approaches, (2)
leveraging unlabeled data in training a CNN-based classifier reduces the
dependency on labeled data by more than 50% to reach the same performance of a
fully supervised CNN, and (3) transferring the knowledge gained from available
labeled data in an external source hospital significantly improves the
performance of a semi-supervised CNN model over their fully supervised
counterparts in a target hospital.
