  This work addresses the problem of semantic scene understanding under fog.
Although marked progress has been made in semantic scene understanding, it is
mainly concentrated on clear-weather scenes. Extending semantic segmentation
methods to adverse weather conditions such as fog is crucial for outdoor
applications. In this paper, we propose a novel method, named Curriculum Model
Adaptation (CMAda), which gradually adapts a semantic segmentation model from
light synthetic fog to dense real fog in multiple steps, using both labeled
synthetic foggy data and unlabeled real foggy data. The method is based on the
fact that the results of semantic segmentation in moderately adverse conditions
(light fog) can be bootstrapped to solve the same problem in highly adverse
conditions (dense fog). CMAda is extensible to other adverse conditions and
provides a new paradigm for learning with synthetic data and unlabeled real
data. In addition, we present three other main stand-alone contributions: 1) a
novel method to add synthetic fog to real, clear-weather scenes using semantic
input; 2) a new fog density estimator; 3) a novel fog densification method to
densify the fog in real foggy scenes without using depth; and 4) the Foggy
Zurich dataset comprising 3808 real foggy images, with pixel-level semantic
annotations for 40 images under dense fog. Our experiments show that 1) our fog
simulation and fog density estimator outperform their state-of-the-art
counterparts with respect to the task of semantic foggy scene understanding
(SFSU); 2) CMAda improves the performance of state-of-the-art models for SFSU
significantly, benefiting both from our synthetic and real foggy data. The
datasets and code are available at the project website.
