  The accuracy and robustness of image classification with supervised deep
learning are dependent on the availability of large-scale, annotated training
data. However, there is a paucity of annotated data available due to the
complexity of manual annotation. To overcome this problem, a popular approach
is to use transferable knowledge across different domains by: 1) using a
generic feature extractor that has been pre-trained on large-scale general
images (i.e., transfer-learned) but which not suited to capture characteristics
from medical images; or 2) fine-tuning generic knowledge with a relatively
smaller number of annotated images. Our aim is to reduce the reliance on
annotated training data by using a new hierarchical unsupervised feature
extractor with a convolutional auto-encoder placed atop of a pre-trained
convolutional neural network. Our approach constrains the rich and generic
image features from the pre-trained domain to a sophisticated representation of
the local image characteristics from the unannotated medical image domain. Our
approach has a higher classification accuracy than transfer-learned approaches
and is competitive with state-of-the-art supervised fine-tuned methods.
