  Commonly employed in polyp segmentation, single image UNet architectures lack
the temporal insight clinicians gain from video data in diagnosing polyps. To
mirror clinical practices more faithfully, our proposed solution,
PolypNextLSTM, leverages video-based deep learning, harnessing temporal
information for superior segmentation performance with the least parameter
overhead, making it possibly suitable for edge devices. PolypNextLSTM employs a
UNet-like structure with ConvNext-Tiny as its backbone, strategically omitting
the last two layers to reduce parameter overhead. Our temporal fusion module, a
Convolutional Long Short Term Memory (ConvLSTM), effectively exploits temporal
features. Our primary novelty lies in PolypNextLSTM, which stands out as the
leanest in parameters and the fastest model, surpassing the performance of five
state-of-the-art image and video-based deep learning models. The evaluation of
the SUN-SEG dataset spans easy-to-detect and hard-to-detect polyp scenarios,
along with videos containing challenging artefacts like fast motion and
occlusion. Comparison against 5 image-based and 5 video-based models
demonstrates PolypNextLSTM's superiority, achieving a Dice score of 0.7898 on
the hard-to-detect polyp test set, surpassing image-based PraNet (0.7519) and
video-based PNSPlusNet (0.7486). Notably, our model excels in videos featuring
complex artefacts such as ghosting and occlusion. PolypNextLSTM, integrating
pruned ConvNext-Tiny with ConvLSTM for temporal fusion, not only exhibits
superior segmentation performance but also maintains the highest frames per
speed among evaluated models. Access code here
https://github.com/mtec-tuhh/PolypNextLSTM
