  The accurate classification of mass lesions in the adrenal glands (adrenal
masses), detected with computed tomography (CT), is important for diagnosis and
patient management. Adrenal masses can be benign or malignant and benign masses
have varying prevalence. Classification methods based on convolutional neural
networks (CNNs) are the state-of-the-art in maximizing inter-class differences
in large medical imaging training datasets. The application of CNNs, to adrenal
masses is challenging due to large intra-class variations, large inter-class
similarities and imbalanced training data due to the size of the mass lesions.
We developed a deep multi-scale resemblance network (DMRN) to overcome these
limitations and leveraged paired CNNs to evaluate the intra-class similarities.
We used multi-scale feature embedding to improve the inter-class separability
by iteratively combining complementary information produced at different scales
of the input to create structured feature descriptors. We augmented the
training data with randomly sampled paired adrenal masses to reduce the
influence of imbalanced training data. We used 229 CT scans of patients with
adrenal masses for evaluation. In a five-fold cross-validation, our method had
the best results (89.52% in accuracy) when compared to the state-of-the-art
methods (p<0.05). We conducted a generalizability analysis of our method on the
ImageCLEF 2016 competition dataset for medical subfigure classification, which
consists of a training set of 6,776 images and a test set of 4,166 images
across 30 classes. Our method achieved better classification performance
(85.90% in accuracy) when compared to the existing methods and was competitive
when compared with methods that require additional training data (1.47% lower
in accuracy). Our DMRN sub-classified adrenal masses on CT and was superior to
state-of-the-art approaches.
