  In this paper, we propose Contextual Guided Segmentation (CGS) framework for
video instance segmentation in three passes. In the first pass, i.e., preview
segmentation, we propose Instance Re-Identification Flow to estimate main
properties of each instance (i.e., human/non-human, rigid/deformable,
known/unknown category) by propagating its preview mask to other frames. In the
second pass, i.e., contextual segmentation, we introduce multiple contextual
segmentation schemes. For human instance, we develop skeleton-guided
segmentation in a frame along with object flow to correct and refine the result
across frames. For non-human instance, if the instance has a wide variation in
appearance and belongs to known categories (which can be inferred from the
initial mask), we adopt instance segmentation. If the non-human instance is
nearly rigid, we train FCNs on synthesized images from the first frame of a
video sequence. In the final pass, i.e., guided segmentation, we develop a
novel fined-grained segmentation method on non-rectangular regions of interest
(ROIs). The natural-shaped ROI is generated by applying guided attention from
the neighbor frames of the current one to reduce the ambiguity in the
segmentation of different overlapping instances. Forward mask propagation is
followed by backward mask propagation to further restore missing instance
fragments due to re-appeared instances, fast motion, occlusion, or heavy
deformation. Finally, instances in each frame are merged based on their depth
values, together with human and non-human object interaction and rare instance
priority. Experiments conducted on the DAVIS Test-Challenge dataset demonstrate
the effectiveness of our proposed framework. We achieved the 3rd consistently
in the DAVIS Challenges 2017-2019 with 75.4%, 72.4%, and 78.4% in terms of
global score, region similarity, and contour accuracy, respectively.
