  With the rapid development of eXplainable Artificial Intelligence (XAI),
perturbation-based XAI algorithms have become quite popular due to their
effectiveness and ease of implementation. The vast majority of
perturbation-based XAI techniques face the challenge of Out-of-Distribution
(OoD) data -- an artifact of randomly perturbed data becoming inconsistent with
the original dataset. OoD data leads to the over-confidence problem in model
predictions, making the existing XAI approaches unreliable. To our best
knowledge, the OoD data problem in perturbation-based XAI algorithms has not
been adequately addressed in the literature. In this work, we address this OoD
data problem by designing an additional module quantifying the affinity between
the perturbed data and the original dataset distribution, which is integrated
into the process of aggregation. Our solution is shown to be compatible with
the most popular perturbation-based XAI algorithms, such as RISE, OCCLUSION,
and LIME. Experiments have confirmed that our methods demonstrate a significant
improvement in general cases using both computational and cognitive metrics.
Especially in the case of degradation, our proposed approach demonstrates
outstanding performance comparing to baselines. Besides, our solution also
resolves a fundamental problem with the faithfulness indicator, a commonly used
evaluation metric of XAI algorithms that appears to be sensitive to the OoD
issue.
