  Asking clarification questions is an active area of research; however,
resources for training and evaluating search clarification methods are not
sufficient. To address this issue, we describe MIMICS-Duo, a new freely
available dataset of 306 search queries with multiple clarifications (a total
of 1,034 query-clarification pairs). MIMICS-Duo contains fine-grained
annotations on clarification questions and their candidate answers and enhances
the existing MIMICS datasets by enabling multi-dimensional evaluation of search
clarification methods, including online and offline evaluation. We conduct
extensive analysis to demonstrate the relationship between offline and online
search clarification datasets and outline several research directions enabled
by MIMICS-Duo. We believe that this resource will help researchers better
understand clarification in search.
