  Deep learning methods have been very effective for a variety of medical
diagnostic tasks and has even beaten human experts on some of those. However,
the black-box nature of the algorithms has restricted clinical use. Recent
explainability studies aim to show the features that influence the decision of
a model the most. The majority of literature reviews of this area have focused
on taxonomy, ethics, and the need for explanations. A review of the current
applications of explainable deep learning for different medical imaging tasks
is presented here. The various approaches, challenges for clinical deployment,
and the areas requiring further research are discussed here from a practical
standpoint of a deep learning researcher designing a system for the clinical
end-users.
