  Trigger warnings are labels that preface documents with sensitive content if
this content could be perceived as harmful by certain groups of readers. Since
warnings about a document intuitively need to be shown before reading it,
authors usually assign trigger warnings at the document level. What parts of
their writing prompted them to assign a warning, however, remains unclear. We
investigate for the first time the feasibility of identifying the triggering
passages of a document, both manually and computationally. We create a dataset
of 4,135 English passages, each annotated with one of eight common trigger
warnings. In a large-scale evaluation, we then systematically evaluate the
effectiveness of fine-tuned and few-shot classifiers, and their
generalizability. We find that trigger annotation belongs to the group of
subjective annotation tasks in NLP, and that automatic trigger classification
remains challenging but feasible.
