  Human visual recognition system shows astonishing capability of compressing
visual information into a set of tokens containing rich representations without
label supervision. One critical driving principle behind it is perceptual
grouping. Despite being widely used in computer vision in the early 2010s, it
remains a mystery whether perceptual grouping can be leveraged to derive a
neural visual recognition backbone that generates as powerful representations.
In this paper, we propose the Perceptual Group Tokenizer, a model that entirely
relies on grouping operations to extract visual features and perform
self-supervised representation learning, where a series of grouping operations
are used to iteratively hypothesize the context for pixels or superpixels to
refine feature representations. We show that the proposed model can achieve
competitive performance compared to state-of-the-art vision architectures, and
inherits desirable properties including adaptive computation without
re-training, and interpretability. Specifically, Perceptual Group Tokenizer
achieves 80.3% on ImageNet-1K self-supervised learning benchmark with linear
probe evaluation, marking a new progress under this paradigm.
