  We present the Multi-Modal Discussion Transformer (mDT), a novel methodfor
detecting hate speech in online social networks such as Reddit discussions. In
contrast to traditional comment-only methods, our approach to labelling a
comment as hate speech involves a holistic analysis of text and images grounded
in the discussion context. This is done by leveraging graph transformers to
capture the contextual relationships in the discussion surrounding a comment
and grounding the interwoven fusion layers that combine text and image
embeddings instead of processing modalities separately. To evaluate our work,
we present a new dataset, HatefulDiscussions, comprising complete multi-modal
discussions from multiple online communities on Reddit. We compare the
performance of our model to baselines that only process individual comments and
conduct extensive ablation studies.
