  Explainable models in Artificial Intelligence are often employed to ensure
transparency and accountability of AI systems. The fidelity of the explanations
are dependent upon the algorithms used as well as on the fidelity of the data.
Many real world datasets have missing values that can greatly influence
explanation fidelity. The standard way to deal with such scenarios is
imputation. This can, however, lead to situations where the imputed values may
correspond to a setting which refer to counterfactuals. Acting on explanations
from AI models with imputed values may lead to unsafe outcomes. In this paper,
we explore different settings where AI models with imputation can be
problematic and describe ways to address such scenarios.
