  Users often have trouble formulating their information needs into words on
the first try when searching online. This can lead to frustration, as they may
have to reformulate their queries when retrieved information is not relevant.
This can be due to a lack of familiarity with the specific terminology related
to their search topic, or because queries are ambiguous and related to multiple
topics. Most modern search engines have interactive features that suggest
clarifications or similar queries based on what others have searched for.
However, the proposed models are either based on a single interaction or
evaluated on search logs, hindering the naturalness of the interactions. In
this paper, we introduce CIRCLE, a generative model for multi-turn query
Clarifications wIth ReinforCement LEarning that leverages multi-turn
interactions through a user simulation framework. Our model aims at generating
a diverse set of query clarifications using a pretrained language model
fine-tuned using reinforcement learning. We evaluate it against well
established google suggestions using a user simulation framework.
