  Few-Shot Segmentation (FSS) aims to segment novel classes using only a few
annotated images. Despite considerable process under pixel-wise support
annotation, current FSS methods still face three issues: the inflexibility of
backbone upgrade without re-training, the inability to uniformly handle various
types of annotations (e.g., scribble, bounding box, mask and text), and the
difficulty in accommodating different annotation quantity. To address these
issues simultaneously, we propose DiffUp, a novel FSS method that
conceptualizes the FSS task as a conditional generative problem using a
diffusion process. For the first issue, we introduce a backbone-agnostic
feature transformation module that converts different segmentation cues into
unified coarse priors, facilitating seamless backbone upgrade without
re-training. For the second issue, due to the varying granularity of
transformed priors from diverse annotation types, we conceptualize these
multi-granular transformed priors as analogous to noisy intermediates at
different steps of a diffusion model. This is implemented via a
self-conditioned modulation block coupled with a dual-level quality modulation
branch. For the third issue, we incorporates an uncertainty-aware information
fusion module that harmonizing the variability across zero-shot, one-shot and
many-shot scenarios. Evaluated through rigorous benchmarks, DiffUp
significantly outperforms existing FSS models in terms of flexibility and
accuracy.
