  Unsupervised anomaly detection in medical images such as chest radiographs is
stepping into the spotlight as it mitigates the scarcity of the labor-intensive
and costly expert annotation of anomaly data. However, nearly all existing
methods are formulated as a one-class classification trained only on
representations from the normal class and discard a potentially significant
portion of the unlabeled data. This paper focuses on a more practical setting,
dual distribution anomaly detection for chest X-rays, using the entire training
data, including both normal and unlabeled images. Inspired by a modern
self-supervised vision transformer model trained using partial image inputs to
reconstruct missing image regions -- we propose AMAE, a two-stage algorithm for
adaptation of the pre-trained masked autoencoder (MAE). Starting from MAE
initialization, AMAE first creates synthetic anomalies from only normal
training images and trains a lightweight classifier on frozen transformer
features. Subsequently, we propose an adaptation strategy to leverage unlabeled
images containing anomalies. The adaptation scheme is accomplished by assigning
pseudo-labels to unlabeled images and using two separate MAE based modules to
model the normative and anomalous distributions of pseudo-labeled images. The
effectiveness of the proposed adaptation strategy is evaluated with different
anomaly ratios in an unlabeled training set. AMAE leads to consistent
performance gains over competing self-supervised and dual distribution anomaly
detection methods, setting the new state-of-the-art on three public chest X-ray
benchmarks: RSNA, NIH-CXR, and VinDr-CXR.
