  In the current context where online platforms have been effectively
weaponized in a variety of geo-political events and social issues, Internet
memes make fair content moderation at scale even more difficult. Existing work
on meme classification and tracking has focused on black-box methods that do
not explicitly consider the semantics of the memes or the context of their
creation. In this paper, we pursue a modular and explainable architecture for
Internet meme understanding. We design and implement multimodal classification
methods that perform example- and prototype-based reasoning over training
cases, while leveraging both textual and visual SOTA models to represent the
individual cases. We study the relevance of our modular and explainable models
in detecting harmful memes on two existing tasks: Hate Speech Detection and
Misogyny Classification. We compare the performance between example- and
prototype-based methods, and between text, vision, and multimodal models,
across different categories of harmfulness (e.g., stereotype and
objectification). We devise a user-friendly interface that facilitates the
comparative analysis of examples retrieved by all of our models for any given
meme, informing the community about the strengths and limitations of these
explainable methods.
