  Albeit with varying degrees of progress in the field of Semi-Supervised
Semantic Segmentation, most of its recent successes are involved in unwieldy
models and the lightweight solution is still not yet explored. We find that
existing knowledge distillation techniques pay more attention to pixel-level
concepts from labeled data, which fails to take more informative cues within
unlabeled data into account. Consequently, we offer the first attempt to
provide lightweight SSSS models via a novel multi-granularity distillation
(MGD) scheme, where multi-granularity is captured from three aspects: i)
complementary teacher structure; ii) labeled-unlabeled data cooperative
distillation; iii) hierarchical and multi-levels loss setting. Specifically,
MGD is formulated as a labeled-unlabeled data cooperative distillation scheme,
which helps to take full advantage of diverse data characteristics that are
essential in the semi-supervised setting. Image-level semantic-sensitive loss,
region-level content-aware loss, and pixel-level consistency loss are set up to
enrich hierarchical distillation abstraction via structurally complementary
teachers. Experimental results on PASCAL VOC2012 and Cityscapes reveal that MGD
can outperform the competitive approaches by a large margin under diverse
partition protocols. For example, the performance of ResNet-18 and MobileNet-v2
backbone is boosted by 11.5% and 4.6% respectively under 1/16 partition
protocol on Cityscapes. Although the FLOPs of the model backbone is compressed
by 3.4-5.3x (ResNet-18) and 38.7-59.6x (MobileNetv2), the model manages to
achieve satisfactory segmentation results.
