  This work investigates the use of interactively updated label suggestions to
improve upon the efficiency of gathering annotations on the task of opinion
mining in German Covid-19 social media data. We develop guidelines to conduct a
controlled annotation study with social science students and find that
suggestions from a model trained on a small, expert-annotated dataset already
lead to a substantial improvement - in terms of inter-annotator agreement(+.14
Fleiss' $\kappa$) and annotation quality - compared to students that do not
receive any label suggestions. We further find that label suggestions from
interactively trained models do not lead to an improvement over suggestions
from a static model. Nonetheless, our analysis of suggestion bias shows that
annotators remain capable of reflecting upon the suggested label in general.
Finally, we confirm the quality of the annotated data in transfer learning
experiments between different annotator groups. To facilitate further research
in opinion mining on social media data, we release our collected data
consisting of 200 expert and 2,785 student annotations.
