  Explanation of AI, as well as fairness of algorithms' decisions and the
transparency of the decision model, are becoming more and more important. And
it is crucial to design effective and human-friendly techniques when opening
the black-box model. Counterfactual conforms to the human way of thinking and
provides a human-friendly explanation, and its corresponding explanation
algorithm refers to a strategic alternation of a given data point so that its
model output is "counter-facted", i.e. the prediction is reverted. In this
paper, we adapt counterfactual explanation over fine-grained image
classification problem. We demonstrated an adaptive method that could give a
counterfactual explanation by showing the composed counterfactual feature map
using top-down layer searching algorithm (TDLS). We have proved that our TDLS
algorithm could provide more flexible counterfactual visual explanation in an
efficient way using VGG-16 model on Caltech-UCSD Birds 200 dataset. At the end,
we discussed several applicable scenarios of counterfactual visual
explanations.
