  Explaining the predictions made by complex machine learning models helps
users to understand and accept the predicted outputs with confidence. One
promising way is to use similarity-based explanation that provides similar
instances as evidence to support model predictions. Several relevance metrics
are used for this purpose. In this study, we investigated relevance metrics
that can provide reasonable explanations to users. Specifically, we adopted
three tests to evaluate whether the relevance metrics satisfy the minimal
requirements for similarity-based explanation. Our experiments revealed that
the cosine similarity of the gradients of the loss performs best, which would
be a recommended choice in practice. In addition, we showed that some metrics
perform poorly in our tests and analyzed the reasons of their failure. We
expect our insights to help practitioners in selecting appropriate relevance
metrics and also aid further researches for designing better relevance metrics
for explanations.
