  Existing multi-stage clustering methods independently learn the salient
features from multiple views and then perform the clustering task.
Particularly, multi-view clustering (MVC) has attracted a lot of attention in
multi-view or multi-modal scenarios. MVC aims at exploring common semantics and
pseudo-labels from multiple views and clustering in a self-supervised manner.
However, limited by noisy data and inadequate feature learning, such a
clustering paradigm generates overconfident pseudo-labels that mis-guide the
model to produce inaccurate predictions. Therefore, it is desirable to have a
method that can correct this pseudo-label mistraction in multi-stage clustering
to avoid the bias accumulation. To alleviate the effect of overconfident
pseudo-labels and improve the generalization ability of the model, this paper
proposes a novel multi-stage deep MVC framework where multi-view
self-distillation (DistilMVC) is introduced to distill dark knowledge of label
distribution. Specifically, in the feature subspace at different hierarchies,
we explore the common semantics of multiple views through contrastive learning
and obtain pseudo-labels by maximizing the mutual information between views.
Additionally, a teacher network is responsible for distilling pseudo-labels
into dark knowledge, supervising the student network and improving its
predictive capabilities to enhance the robustness. Extensive experiments on
real-world multi-view datasets show that our method has better clustering
performance than state-of-the-art methods.
