  Conversational Question Answering (CQA) aims to answer questions contained
within dialogues, which are not easily interpretable without context.
Developing a model to rewrite conversational questions into self-contained ones
is an emerging solution in industry settings as it allows using existing
single-turn QA systems to avoid training a CQA model from scratch. Previous
work trains rewriting models using human rewrites as supervision. However, such
objectives are disconnected with QA models and therefore more human-like
rewrites do not guarantee better QA performance. In this paper we propose using
QA feedback to supervise the rewriting model with reinforcement learning.
Experiments show that our approach can effectively improve QA performance over
baselines for both extractive and retrieval QA. Furthermore, human evaluation
shows that our method can generate more accurate and detailed rewrites when
compared to human annotations.
