  We present a methodology to systematically test conversational recommender
systems with regards to conversational breakdowns. It involves examining
conversations generated between the system and simulated users for a set of
pre-defined breakdown types, extracting responsible conversational paths, and
characterizing them in terms of the underlying dialogue intents. User
simulation offers the advantages of simplicity, cost-effectiveness, and time
efficiency for obtaining conversations where potential breakdowns can be
identified. The proposed methodology can be used as diagnostic tool as well as
a development tool to improve conversational recommendation systems. We apply
our methodology in a case study with an existing conversational recommender
system and user simulator, demonstrating that with just a few iterations, we
can make the system more robust to conversational breakdowns.
