  Conversational search aims to retrieve passages containing essential
information to answer queries in a multi-turn conversation. In conversational
search, reformulating context-dependent conversational queries into stand-alone
forms is imperative to effectively utilize off-the-shelf retrievers. Previous
methodologies for conversational query reformulation frequently depend on
human-annotated rewrites. However, these manually crafted queries often result
in sub-optimal retrieval performance and require high collection costs. To
address these challenges, we propose Iterative Conversational Query
Reformulation (IterCQR), a methodology that conducts query reformulation
without relying on human rewrites. IterCQR iteratively trains the
conversational query reformulation (CQR) model by directly leveraging
information retrieval (IR) signals as a reward. Our IterCQR training guides the
CQR model such that generated queries contain necessary information from the
previous dialogue context. Our proposed method shows state-of-the-art
performance on two widely-used datasets, demonstrating its effectiveness on
both sparse and dense retrievers. Moreover, IterCQR exhibits superior
performance in challenging settings such as generalization on unseen datasets
and low-resource scenarios.
