  Filtering and annotating textual data are routine tasks in many areas, like
social media or news analytics. Automating these tasks allows to scale the
analyses wrt. speed and breadth of content covered and decreases the manual
effort required. Due to technical advancements in Natural Language Processing,
specifically the success of large foundation models, a new tool for automating
such annotation processes by using a text-to-text interface given written
guidelines without providing training samples has become available.
  In this work, we assess these advancements in-the-wild by empirically testing
them in an annotation task on German Twitter data about social and political
European crises. We compare the prompt-based results with our human annotation
and preceding classification approaches, including Naive Bayes and a BERT-based
fine-tuning/domain adaptation pipeline. Our results show that the prompt-based
approach - despite being limited by local computation resources during the
model selection - is comparable with the fine-tuned BERT but without any
annotated training data. Our findings emphasize the ongoing paradigm shift in
the NLP landscape, i.e., the unification of downstream tasks and elimination of
the need for pre-labeled training data.
