  Post-hoc analysis is a popular category in eXplainable artificial
intelligence (XAI) study. In particular, methods that generate heatmaps have
been used to explain the deep neural network (DNN), a black-box model. Heatmaps
can be appealing due to the intuitive and visual ways to understand them but
assessing their qualities might not be straightforward. Different ways to
assess heatmaps' quality have their own merits and shortcomings. This paper
introduces a synthetic dataset that can be generated adhoc along with the
ground-truth heatmaps for more objective quantitative assessment. Each sample
data is an image of a cell with easily recognized features that are
distinguished from localization ground-truth mask, hence facilitating a more
transparent assessment of different XAI methods. Comparison and recommendations
are made, shortcomings are clarified along with suggestions for future research
directions to handle the finer details of select post-hoc analysis methods.
Furthermore, mabCAM is introduced as the heatmap generation method compatible
with our ground-truth heatmaps. The framework is easily generalizable and uses
only standard deep learning components.
