  The rapid development of Vision Foundation Model (VFM) brings inherent
out-domain generalization for a variety of down-stream tasks. Among them,
domain generalized semantic segmentation (DGSS) holds unique challenges as the
cross-domain images share common pixel-wise content information but vary
greatly in terms of the style. In this paper, we present a novel
Spectral-dEcomposed Token (SET) learning framework to advance the frontier.
Delving into further than existing fine-tuning token & frozen backbone
paradigm, the proposed SET especially focuses on the way learning
style-invariant features from these learnable tokens. Particularly, the frozen
VFM features are first decomposed into the phase and amplitude components in
the frequency space, which mainly contain the information of content and style,
respectively, and then separately processed by learnable tokens for
task-specific information extraction. After the decomposition, style variation
primarily impacts the token-based feature enhancement within the amplitude
branch. To address this issue, we further develop an attention optimization
method to bridge the gap between style-affected representation and static
tokens during inference. Extensive cross-domain experiments show its
state-of-the-art performance.
