  Providing explanations is considered an imperative ability for an AI agent in
a human-robot teaming framework. The right explanation provides the rationale
behind an AI agent's decision-making. However, to maintain the human teammate's
cognitive demand to comprehend the provided explanations, prior works have
focused on providing explanations in a specific order or intertwining the
explanation generation with plan execution. Moreover, these approaches do not
consider the degree of details required to share throughout the provided
explanations. In this work, we argue that the agent-generated explanations,
especially the complex ones, should be abstracted to be aligned with the level
of details the human teammate desires to maintain the recipient's cognitive
load. Therefore, learning a hierarchical explanations model is a challenging
task. Moreover, the agent needs to follow a consistent high-level policy to
transfer the learned teammate preferences to a new scenario while lower-level
detailed plans are different. Our evaluation confirmed the process of
understanding an explanation, especially a complex and detailed explanation, is
hierarchical. The human preference that reflected this aspect corresponded
exactly to creating and employing abstraction for knowledge assimilation hidden
deeper in our cognitive process. We showed that hierarchical explanations
achieved better task performance and behavior interpretability while reduced
cognitive load. These results shed light on designing explainable agents
utilizing reinforcement learning and planning across various domains.
