  Few-shot segmentation (FSS) aims at performing semantic segmentation on novel
classes given a few annotated support samples. With a rethink of recent
advances, we find that the current FSS framework has deviated far from the
supervised segmentation framework: Given the deep features, FSS methods
typically use an intricate decoder to perform sophisticated pixel-wise
matching, while the supervised segmentation methods use a simple linear
classification head. Due to the intricacy of the decoder and its matching
pipeline, it is not easy to follow such an FSS framework. This paper revives
the straightforward framework of "feature extractor $+$ linear classification
head" and proposes a novel Feature-Proxy Transformer (FPTrans) method, in which
the "proxy" is the vector representing a semantic class in the linear
classification head. FPTrans has two keypoints for learning discriminative
features and representative proxies: 1) To better utilize the limited support
samples, the feature extractor makes the query interact with the support
features from the bottom to top layers using a novel prompting strategy. 2)
FPTrans uses multiple local background proxies (instead of a single one)
because the background is not homogeneous and may contain some novel foreground
regions. These two keypoints are easily integrated into the vision transformer
backbone with the prompting mechanism in the transformer. Given the learned
features and proxies, FPTrans directly compares their cosine similarity for
segmentation. Although the framework is straightforward, we show that FPTrans
achieves competitive FSS accuracy on par with state-of-the-art decoder-based
methods.
