  We study continually improving an extractive question answering (QA) system
via human user feedback. We design and deploy an iterative approach, where
information-seeking users ask questions, receive model-predicted answers, and
provide feedback. We conduct experiments involving thousands of user
interactions under diverse setups to broaden the understanding of learning from
feedback over time. Our experiments show effective improvement from user
feedback of extractive QA models over time across different data regimes,
including significant potential for domain adaptation.
