  We tackle the problem of novel class discovery, which aims to learn novel
classes without supervision based on labeled data from known classes. A key
challenge lies in transferring the knowledge in the known-class data to the
learning of novel classes. Previous methods mainly focus on building a shared
representation space for knowledge transfer and often ignore modeling class
relations. To address this, we introduce a class relation representation for
the novel classes based on the predicted class distribution of a model trained
on known classes. Empirically, we find that such class relation becomes less
informative during typical discovery training. To prevent such information
loss, we propose a novel knowledge distillation framework, which utilizes our
class-relation representation to regularize the learning of novel classes. In
addition, to enable a flexible knowledge distillation scheme for each data
point in novel classes, we develop a learnable weighting function for the
regularization, which adaptively promotes knowledge transfer based on the
semantic similarity between the novel and known classes. To validate the
effectiveness and generalization of our method, we conduct extensive
experiments on multiple benchmarks, including CIFAR100, Stanford Cars, CUB, and
FGVC-Aircraft datasets. Our results demonstrate that the proposed method
outperforms the previous state-of-the-art methods by a significant margin on
almost all benchmarks. Code is available at
\href{https://github.com/kleinzcy/Cr-KD-NCD}{here}.
