  Modern deep learning models are notoriously opaque, which has motivated the
development of methods for interpreting how deep models predict. This goal is
usually approached with attribution method, which assesses the influence of
features on model predictions. As an explanation method, the evaluation
criteria of attribution methods is how accurately it re-reflects the actual
reasoning process of the model (faithfulness). Meanwhile, since the reasoning
process of deep models is inaccessible, researchers design various evaluation
methods to demonstrate their arguments. However, some crucial logic traps in
these evaluation methods are ignored in most works, causing inaccurate
evaluation and unfair comparison. This paper systematically reviews existing
methods for evaluating attribution scores and summarizes the logic traps in
these methods. We further conduct experiments to demonstrate the existence of
each logic trap. Through both the theoretical and experimental analysis, we
hope to increase attention on the inaccurate evaluation of attribution scores.
Moreover, with this paper, we suggest stopping focusing on improving
performance under unreliable evaluation systems and starting efforts on
reducing the impact of proposed logic traps
