  Image classification is central to the big data revolution in medicine.
Improved information processing methods for diagnosis and classification of
digital medical images have shown to be successful via deep learning
approaches. As this field is explored, there are limitations to the performance
of traditional supervised classifiers. This paper outlines an approach that is
different from the current medical image classification tasks that view the
issue as multi-class classification. We performed a hierarchical classification
using our Hierarchical Medical Image classification (HMIC) approach. HMIC uses
stacks of deep learning models to give particular comprehension at each level
of the clinical picture hierarchy. For testing our performance, we use biopsy
of the small bowel images that contain three categories in the parent level
(Celiac Disease, Environmental Enteropathy, and histologically normal
controls). For the child level, Celiac Disease Severity is classified into 4
classes (I, IIIa, IIIb, and IIIC).
