  Research in interpretable machine learning proposes different computational
and human subject approaches to evaluate model saliency explanations. These
approaches measure different qualities of explanations to achieve diverse goals
in designing interpretable machine learning systems. In this paper, we propose
a human attention benchmark for image and text domains using multi-layer human
attention masks aggregated from multiple human annotators. We then present an
evaluation study to evaluate model saliency explanations obtained using
Grad-cam and LIME techniques. We demonstrate our benchmark's utility for
quantitative evaluation of model explanations by comparing it with human
subjective ratings and ground-truth single-layer segmentation masks
evaluations. Our study results show that our threshold agnostic evaluation
method with the human attention baseline is more effective than single-layer
object segmentation masks to ground truth. Our experiments also reveal user
biases in the subjective rating of model saliency explanations.
