  The introduction of the MUStARD dataset, and its emotion recognition
extension MUStARD++, have identified sarcasm to be a multi-modal phenomenon --
expressed not only in natural language text, but also through manners of speech
(like tonality and intonation) and visual cues (facial expression). With this
work, we aim to perform a rigorous benchmarking of the MUStARD++ dataset by
considering state-of-the-art language, speech, and visual encoders, for fully
utilizing the totality of the multi-modal richness that it has to offer,
achieving a 2\% improvement in macro-F1 over the existing benchmark.
Additionally, to cure the imbalance in the `sarcasm type' category in
MUStARD++, we propose an extension, which we call \emph{MUStARD++ Balanced},
benchmarking the same with instances from the extension split across both train
and test sets, achieving a further 2.4\% macro-F1 boost. The new clips were
taken from a novel source -- the TV show, House MD, which adds to the diversity
of the dataset, and were manually annotated by multiple annotators with
substantial inter-annotator agreement in terms of Cohen's kappa and
Krippendorf's alpha. Our code, extended data, and SOTA benchmark models are
made public.
