  Few-shot segmentation aims to train a segmentation model that can fast adapt
to novel classes with few exemplars. The conventional training paradigm is to
learn to make predictions on query images conditioned on the features from
support images. Previous methods only utilized the semantic-level prototypes of
support images as conditional information. These methods cannot utilize all
pixel-wise support information for the query predictions, which is however
critical for the segmentation task. In this paper, we focus on utilizing
pixel-wise relationships between support and query images to facilitate the
few-shot segmentation task. We design a novel Cycle-Consistent TRansformer
(CyCTR) module to aggregate pixel-wise support features into query ones. CyCTR
performs cross-attention between features from different images, i.e. support
and query images. We observe that there may exist unexpected irrelevant
pixel-level support features. Directly performing cross-attention may aggregate
these features from support to query and bias the query features. Thus, we
propose using a novel cycle-consistent attention mechanism to filter out
possible harmful support features and encourage query features to attend to the
most informative pixels from support images. Experiments on all few-shot
segmentation benchmarks demonstrate that our proposed CyCTR leads to remarkable
improvement compared to previous state-of-the-art methods. Specifically, on
Pascal-$5^i$ and COCO-$20^i$ datasets, we achieve 67.5% and 45.6% mIoU for
5-shot segmentation, outperforming previous state-of-the-art methods by 5.6%
and 7.1% respectively.
