  Deep visual models have widespread applications in high-stake domains. Hence,
their black-box nature is currently attracting a large interest of the research
community. We present the first survey in Explainable AI that focuses on the
methods and metrics for interpreting deep visual models. Covering the landmark
contributions along the state-of-the-art, we not only provide a taxonomic
organization of the existing techniques, but also excavate a range of
evaluation metrics and collate them as measures of different properties of
model explanations. Along the insightful discussion on the current trends, we
also discuss the challenges and future avenues for this research direction.
