  Deep spectral methods reframe the image decomposition process as a graph
partitioning task by extracting features using self-supervised learning and
utilizing the Laplacian of the affinity matrix to obtain eigensegments.
However, instance segmentation has received less attention compared to other
tasks within the context of deep spectral methods. This paper addresses the
fact that not all channels of the feature map extracted from a self-supervised
backbone contain sufficient information for instance segmentation purposes. In
fact, Some channels are noisy and hinder the accuracy of the task. To overcome
this issue, this paper proposes two channel reduction modules: Noise Channel
Reduction (NCR) and Deviation-based Channel Reduction (DCR). The NCR retains
channels with lower entropy, as they are less likely to be noisy, while DCR
prunes channels with low standard deviation, as they lack sufficient
information for effective instance segmentation. Furthermore, the paper
demonstrates that the dot product, commonly used in deep spectral methods, is
not suitable for instance segmentation due to its sensitivity to feature map
values, potentially leading to incorrect instance segments. A new similarity
metric called Bray-Curtis over Chebyshev (BoC) is proposed to address this
issue. It takes into account the distribution of features in addition to their
values, providing a more robust similarity measure for instance segmentation.
Quantitative and qualitative results on the Youtube-VIS2019 dataset highlight
the improvements achieved by the proposed channel reduction methods and the use
of BoC instead of the conventional dot product for creating the affinity
matrix. These improvements are observed in terms of mean Intersection over
Union and extracted instance segments, demonstrating enhanced instance
segmentation performance. The code is available on:
https://github.com/farnooshar/SpecUnIIS
