  Researchers have proposed various methods for visually interpreting the
Convolutional Neural Network (CNN) via saliency maps, which include
Class-Activation-Map (CAM) based approaches as a leading family. However, in
terms of the internal design logic, existing CAM-based approaches often
overlook the causal perspective that answers the core "why" question to help
humans understand the explanation. Additionally, current CNN explanations lack
the consideration of both necessity and sufficiency, two complementary sides of
a desirable explanation. This paper presents a causality-driven framework,
SUNY, designed to rationalize the explanations toward better human
understanding. Using the CNN model's input features or internal filters as
hypothetical causes, SUNY generates explanations by bi-directional
quantifications on both the necessary and sufficient perspectives. Extensive
evaluations justify that SUNY not only produces more informative and convincing
explanations from the angles of necessity and sufficiency, but also achieves
performances competitive to other approaches across different CNN architectures
over large-scale datasets, including ILSVRC2012 and CUB-200-2011.
