  Indirect speech such as sarcasm achieves a constellation of discourse goals
in human communication. While the indirectness of figurative language warrants
speakers to achieve certain pragmatic goals, it is challenging for AI agents to
comprehend such idiosyncrasies of human communication. Though sarcasm
identification has been a well-explored topic in dialogue analysis, for
conversational systems to truly grasp a conversation's innate meaning and
generate appropriate responses, simply detecting sarcasm is not enough; it is
vital to explain its underlying sarcastic connotation to capture its true
essence. In this work, we study the discourse structure of sarcastic
conversations and propose a novel task - Sarcasm Explanation in Dialogue (SED).
Set in a multimodal and code-mixed setting, the task aims to generate natural
language explanations of satirical conversations. To this end, we curate WITS,
a new dataset to support our task. We propose MAF (Modality Aware Fusion), a
multimodal context-aware attention and global information fusion module to
capture multimodality and use it to benchmark WITS. The proposed attention
module surpasses the traditional multimodal fusion baselines and reports the
best performance on almost all metrics. Lastly, we carry out detailed analyses
both quantitatively and qualitatively.
