  Pathological anomalies exhibit diverse appearances in medical imaging, making
it difficult to collect and annotate a representative amount of data required
to train deep learning models in a supervised setting. Therefore, in this work,
we tackle anomaly detection in medical images training our framework using only
healthy samples. We propose to use the Masked Autoencoder model to learn the
structure of the normal samples, then train an anomaly classifier on top of the
difference between the original image and the reconstruction provided by the
masked autoencoder. We train the anomaly classifier in a supervised manner
using as negative samples the reconstruction of the healthy scans, while as
positive samples, we use pseudo-abnormal scans obtained via our novel
pseudo-abnormal module. The pseudo-abnormal module alters the reconstruction of
the normal samples by changing the intensity of several regions. We conduct
experiments on two medical image data sets, namely BRATS2020 and LUNA16 and
compare our method with four state-of-the-art anomaly detection frameworks,
namely AST, RD4AD, AnoVAEGAN and f-AnoGAN.
