  Techniques that explain the predictions of black-box machine learning models
are crucial to make the models transparent, thereby increasing trust in AI
systems. The input features to the models often have a nested structure that
consists of high- and low-level features, and each high-level feature is
decomposed into multiple low-level features. For such inputs, both high-level
feature attributions (HiFAs) and low-level feature attributions (LoFAs) are
important for better understanding the model's decision. In this paper, we
propose a model-agnostic local explanation method that effectively exploits the
nested structure of the input to estimate the two-level feature attributions
simultaneously. A key idea of the proposed method is to introduce the
consistency property that should exist between the HiFAs and LoFAs, thereby
bridging the separate optimization problems for estimating them. Thanks to this
consistency property, the proposed method can produce HiFAs and LoFAs that are
both faithful to the black-box models and consistent with each other, using a
smaller number of queries to the models. In experiments on image classification
in multiple instance learning and text classification using language models, we
demonstrate that the HiFAs and LoFAs estimated by the proposed method are
accurate, faithful to the behaviors of the black-box models, and provide
consistent explanations.
