  Asking clarifying questions in response to search queries has been recognized
as a useful technique for revealing the underlying intent of the query.
Clarification has applications in retrieval systems with different interfaces,
from the traditional web search interfaces to the limited bandwidth interfaces
as in speech-only and small screen devices. Generation and evaluation of
clarifying questions have been recently studied in the literature. However,
user interaction with clarifying questions is relatively unexplored. In this
paper, we conduct a comprehensive study by analyzing large-scale user
interactions with clarifying questions in a major web search engine. In more
detail, we analyze the user engagements received by clarifying questions based
on different properties of search queries, clarifying questions, and their
candidate answers. We further study click bias in the data, and show that even
though reading clarifying questions and candidate answers does not take
significant efforts, there still exist some position and presentation biases in
the data. We also propose a model for learning representation for clarifying
questions based on the user interaction data as implicit feedback. The model is
used for re-ranking a number of automatically generated clarifying questions
for a given query. Evaluation on both click data and human labeled data
demonstrates the high quality of the proposed method.
