  Traditional anomaly detection methods focus on detecting inter-class
variations while medical image novelty identification is inherently an
intra-class detection problem. For example, a machine learning model trained
with normal chest X-ray and common lung abnormalities, is expected to discover
and flag idiopathic pulmonary fibrosis which a rare lung disease and unseen by
the model during training. The nuances from intra-class variations and lack of
relevant training data in medical image analysis pose great challenges for
existing anomaly detection methods. To tackle the challenges, we propose a
hybrid model - Transformation-based Embedding learning for Novelty Detection
(TEND) which without any out-of-distribution training data, performs novelty
identification by combining both autoencoder-based and classifier-based method.
With a pre-trained autoencoder as image feature extractor, TEND learns to
discriminate the feature embeddings of in-distribution data from the
transformed counterparts as fake out-of-distribution inputs. To enhance the
separation, a distance objective is optimized to enforce a margin between the
two classes. Extensive experimental results on both natural image datasets and
medical image datasets are presented and our method out-performs
state-of-the-art approaches.
