  In Dynamic Adversarial Data Collection (DADC), human annotators are tasked
with finding examples that models struggle to predict correctly. Models trained
on DADC-collected training data have been shown to be more robust in
adversarial and out-of-domain settings, and are considerably harder for humans
to fool. However, DADC is more time-consuming than traditional data collection
and thus more costly per annotated example. In this work, we examine whether we
can maintain the advantages of DADC, without incurring the additional cost. To
that end, we introduce Generative Annotation Assistants (GAAs),
generator-in-the-loop models that provide real-time suggestions that annotators
can either approve, modify, or reject entirely. We collect training datasets in
twenty experimental settings and perform a detailed analysis of this approach
for the task of extractive question answering (QA) for both standard and
adversarial data collection. We demonstrate that GAAs provide significant
efficiency benefits with over a 30% annotation speed-up, while leading to over
a 5x improvement in model fooling rates. In addition, we find that using
GAA-assisted training data leads to higher downstream model performance on a
variety of question answering tasks over adversarial data collection.
