  Image classification is an essential part of computer vision which assigns a
given input image to a specific category based on the similarity evaluation
within given criteria. While promising classifiers can be obtained through deep
learning models, these approaches lack explainability, where the classification
results are hard to interpret in a human-understandable way. In this paper, we
present an explainable metric learning framework, which constructs hierarchical
levels of semantic segments of an image for better interpretability. The key
methodology involves a bottom-up learning strategy, starting by training the
local metric learning model for the individual segments and then combining
segments to compose comprehensive metrics in a tree. Specifically, our approach
enables a more human-understandable similarity measurement between two images
based on the semantic segments within it, which can be utilized to generate new
samples to reduce bias in a training dataset. Extensive experimental evaluation
demonstrates that the proposed approach can drastically improve model accuracy
compared with state-of-the-art methods.
