  The COVID-19 pandemic has made a huge global impact and cost millions of
lives. As COVID-19 vaccines were rolled out, they were quickly met with
widespread hesitancy. To address the concerns of hesitant people, we launched
VIRA, a public dialogue system aimed at addressing questions and concerns
surrounding the COVID-19 vaccines. Here, we release VIRADialogs, a dataset of
over 8k dialogues conducted by actual users with VIRA, providing a unique
real-world conversational dataset. In light of rapid changes in users' intents,
due to updates in guidelines or in response to new information, we highlight
the important task of intent discovery in this use-case. We introduce a novel
automatic evaluation framework for intent discovery, leveraging the existing
intent classifier of VIRA. We use this framework to report baseline intent
discovery results over VIRADialogs, that highlight the difficulty of this task.
