  Systems like Voice-command based conversational agents are characterized by a
pre-defined set of skills or intents to perform user specified tasks. In the
course of time, newer intents may emerge requiring retraining. However, the
newer intents may not be explicitly announced and need to be inferred
dynamically. Thus, there are two important tasks at hand (a). identifying
emerging new intents, (b). annotating data of the new intents so that the
underlying classifier can be retrained efficiently. The tasks become specially
challenging when a large number of new intents emerge simultaneously and there
is a limited budget of manual annotation. In this paper, we propose MNID
(Multiple Novel Intent Detection) which is a cluster based framework to detect
multiple novel intents with budgeted human annotation cost. Empirical results
on various benchmark datasets (of different sizes) demonstrate that MNID, by
intelligently using the budget for annotation, outperforms the baseline methods
in terms of accuracy and F1-score.
