  Automatic and accurate segmentation of colon polyps is essential for early
diagnosis of colorectal cancer. Advanced deep learning models have shown
promising results in polyp segmentation. However, they still have limitations
in representing multi-scale features and generalization capability. To address
these issues, this paper introduces RaBiT, an encoder-decoder model that
incorporates a lightweight Transformer-based architecture in the encoder to
model multiple-level global semantic relationships. The decoder consists of
several bidirectional feature pyramid layers with reverse attention modules to
better fuse feature maps at various levels and incrementally refine polyp
boundaries. We also propose ideas to lighten the reverse attention module and
make it more suitable for multi-class segmentation. Extensive experiments on
several benchmark datasets show that our method outperforms existing methods
across all datasets while maintaining low computational complexity. Moreover,
our method demonstrates high generalization capability in cross-dataset
experiments, even when the training and test sets have different
characteristics.
