  Social media platforms give rise to an abundance of posts and comments on
every topic imaginable. Many of these posts express opinions on various aspects
of society, but their unfalsifiable nature makes them ill-suited to
fact-checking pipelines. In this work, we aim to distill such posts into a
small set of narratives that capture the essential claims related to a given
topic. Understanding and visualizing these narratives can facilitate more
informed debates on social media. As a first step towards systematically
identifying the underlying narratives on social media, we introduce PAPYER, a
fine-grained dataset of online comments related to hygiene in public restrooms,
which contains a multitude of unfalsifiable claims. We present a
human-in-the-loop pipeline that uses a combination of machine and human kernels
to discover the prevailing narratives and show that this pipeline outperforms
recent large transformer models and state-of-the-art unsupervised topic models.
