  It is helpful in preventing colorectal cancer to detect and treat polyps in
the gastrointestinal tract early. However, there have been few studies to date
on designing polyp image classification networks that balance efficiency and
accuracy. This challenge is mainly attributed to the fact that polyps are
similar to other pathologies and have complex features influenced by texture,
color, and morphology. In this paper, we propose a novel network DFE-IANet
based on both spectral transformation and feature interaction. Firstly, to
extract detailed features and multi-scale features, the features are
transformed by the multi-scale frequency domain feature extraction (MSFD) block
to extract texture details at the fine-grained level in the frequency domain.
Secondly, the multi-scale interaction attention (MSIA) block is designed to
enhance the network's capability of extracting critical features. This block
introduces multi-scale features into self-attention, aiming to adaptively guide
the network to concentrate on vital regions. Finally, with a compact parameter
of only 4M, DFE-IANet outperforms the latest and classical networks in terms of
efficiency. Furthermore, DFE-IANet achieves state-of-the-art (SOTA) results on
the challenging Kvasir dataset, demonstrating a remarkable Top-1 accuracy of
93.94%. This outstanding accuracy surpasses ViT by 8.94%, ResNet50 by 1.69%,
and VMamba by 1.88%. Our code is publicly available at
https://github.com/PURSUETHESUN/DFE-IANet.
