  Explanations form the foundation of knowledge sharing and build upon
communication principles, social dynamics, and learning theories. We focus
specifically on conversational approaches for explanations because the context
is highly adaptive and interactive. Our research leverages previous work on
explanatory acts, a framework for understanding the different strategies that
explainers and explainees employ in a conversation to both explain, understand,
and engage with the other party. We use the 5-Levels dataset was constructed
from the WIRED YouTube series by Wachsmuth et al., and later annotated by
Booshehri et al. with explanatory acts. These annotations provide a framework
for understanding how explainers and explainees structure their response when
crafting a response.
  With the rise of generative AI in the past year, we hope to better understand
the capabilities of Large Language Models (LLMs) and how they can augment
expert explainer's capabilities in conversational settings. To achieve this
goal, the 5-Levels dataset (We use Booshehri et al.'s 2023 annotated dataset
with explanatory acts.) allows us to audit the ability of LLMs in engaging in
explanation dialogues. To evaluate the effectiveness of LLMs in generating
explainer responses, we compared 3 different strategies, we asked human
annotators to evaluate 3 different strategies: human explainer response, GPT4
standard response, GPT4 response with Explanation Moves.
