  We present GvSeg, a general video segmentation framework for addressing four
different video segmentation tasks (i.e., instance, semantic, panoptic, and
exemplar-guided) while maintaining an identical architectural design.
Currently, there is a trend towards developing general video segmentation
solutions that can be applied across multiple tasks. This streamlines research
endeavors and simplifies deployment. However, such a highly homogenized
framework in current design, where each element maintains uniformity, could
overlook the inherent diversity among different tasks and lead to suboptimal
performance. To tackle this, GvSeg: i) provides a holistic disentanglement and
modeling for segment targets, thoroughly examining them from the perspective of
appearance, position, and shape, and on this basis, ii) reformulates the query
initialization, matching and sampling strategies in alignment with the
task-specific requirement. These architecture-agnostic innovations empower
GvSeg to effectively address each unique task by accommodating the specific
properties that characterize them. Extensive experiments on seven gold-standard
benchmark datasets demonstrate that GvSeg surpasses all existing
specialized/general solutions by a significant margin on four different video
segmentation tasks.
