  Users in consumption domains, like music, are often able to more efficiently
provide preferences over a set of items (e.g. a playlist or radio) than over
single items (e.g. songs). Unfortunately, this is an underexplored area of
research, with most existing recommendation systems limited to understanding
preferences over single items. Curating an item set exponentiates the search
space that recommender systems must consider (all subsets of items!): this
motivates conversational approaches-where users explicitly state or refine
their preferences and systems elicit preferences in natural language-as an
efficient way to understand user needs. We call this task conversational item
set curation and present a novel data collection methodology that efficiently
collects realistic preferences about item sets in a conversational setting by
observing both item-level and set-level feedback. We apply this methodology to
music recommendation to build the Conversational Playlist Curation Dataset
(CPCD), where we show that it leads raters to express preferences that would
not be otherwise expressed. Finally, we propose a wide range of conversational
retrieval models as baselines for this task and evaluate them on the dataset.
