  Contextual information plays a core role for video semantic segmentation
(VSS). This paper summarizes contexts for VSS in two-fold: local temporal
contexts (LTC) which define the contexts from neighboring frames, and global
temporal contexts (GTC) which represent the contexts from the whole video. As
for LTC, it includes static and motional contexts, corresponding to static and
moving content in neighboring frames, respectively. Previously, both static and
motional contexts have been studied. However, there is no research about
simultaneously learning static and motional contexts (highly complementary).
Hence, we propose a Coarse-to-Fine Feature Mining (CFFM) technique to learn a
unified presentation of LTC. CFFM contains two parts: Coarse-to-Fine Feature
Assembling (CFFA) and Cross-frame Feature Mining (CFM). CFFA abstracts static
and motional contexts, and CFM mines useful information from nearby frames to
enhance target features. To further exploit more temporal contexts, we propose
CFFM++ by additionally learning GTC from the whole video. Specifically, we
uniformly sample certain frames from the video and extract global contextual
prototypes by k-means. The information within those prototypes is mined by CFM
to refine target features. Experimental results on popular benchmarks
demonstrate that CFFM and CFFM++ perform favorably against state-of-the-art
methods. Our code is available at https://github.com/GuoleiSun/VSS-CFFM
