  Novelty Detection methods identify samples that are not representative of a
model's training set thereby flagging misleading predictions and bringing a
greater flexibility and transparency at deployment time. However, research in
this area has only considered Novelty Detection in the offline setting.
Recently, there has been a growing realization in the computer vision community
that applications demand a more flexible framework - Continual Learning - where
new batches of data representing new domains, new classes or new tasks become
available at different points in time. In this setting, Novelty Detection
becomes more important, interesting and challenging. This work identifies the
crucial link between the two problems and investigates the Novelty Detection
problem under the Continual Learning setting. We formulate the Continual
Novelty Detection problem and present a benchmark, where we compare several
Novelty Detection methods under different Continual Learning settings.
  We show that Continual Learning affects the behaviour of novelty detection
algorithms, while novelty detection can pinpoint insights in the behaviour of a
continual learner. We further propose baselines and discuss possible research
directions. We believe that the coupling of the two problems is a promising
direction to bring vision models into practice.
