  Open-Vocabulary Video Instance Segmentation (VIS) is attracting increasing
attention due to its ability to segment and track arbitrary objects. However,
the recent Open-Vocabulary VIS attempts obtained unsatisfactory results,
especially in terms of generalization ability of novel categories. We discover
that the domain gap between the VLM features (e.g., CLIP) and the instance
queries and the underutilization of temporal consistency are two central
causes. To mitigate these issues, we design and train a novel Open-Vocabulary
VIS baseline called OVFormer. OVFormer utilizes a lightweight module for
unified embedding alignment between query embeddings and CLIP image embeddings
to remedy the domain gap. Unlike previous image-based training methods, we
conduct video-based model training and deploy a semi-online inference scheme to
fully mine the temporal consistency in the video. Without bells and whistles,
OVFormer achieves 21.9 mAP with a ResNet-50 backbone on LV-VIS, exceeding the
previous state-of-the-art performance by 7.7. Extensive experiments on some
Close-Vocabulary VIS datasets also demonstrate the strong zero-shot
generalization ability of OVFormer (+ 7.6 mAP on YouTube-VIS 2019, + 3.9 mAP on
OVIS). Code is available at https://github.com/fanghaook/OVFormer.
