  Feature attribution a.k.a. input salience methods which assign an importance
score to a feature are abundant but may produce surprisingly different results
for the same model on the same input. While differences are expected if
disparate definitions of importance are assumed, most methods claim to provide
faithful attributions and point at the features most relevant for a model's
prediction. Existing work on faithfulness evaluation is not conclusive and does
not provide a clear answer as to how different methods are to be compared.
Focusing on text classification and the model debugging scenario, our main
contribution is a protocol for faithfulness evaluation that makes use of
partially synthetic data to obtain ground truth for feature importance ranking.
Following the protocol, we do an in-depth analysis of four standard salience
method classes on a range of datasets and shortcuts for BERT and LSTM models
and demonstrate that some of the most popular method configurations provide
poor results even for simplest shortcuts. We recommend following the protocol
for each new task and model combination to find the best method for identifying
shortcuts.
