  Existing techniques to adapt semantic segmentation networks across the source
and target domains within deep convolutional neural networks (CNNs) deal with
all the samples from the two domains in a global or category-aware manner. They
do not consider an inter-class variation within the target domain itself or
estimated category, providing the limitation to encode the domains having a
multi-modal data distribution. To overcome this limitation, we introduce a
learnable clustering module, and a novel domain adaptation framework called
cross-domain grouping and alignment. To cluster the samples across domains with
an aim to maximize the domain alignment without forgetting precise segmentation
ability on the source domain, we present two loss functions, in particular, for
encouraging semantic consistency and orthogonality among the clusters. We also
present a loss so as to solve a class imbalance problem, which is the other
limitation of the previous methods. Our experiments show that our method
consistently boosts the adaptation performance in semantic segmentation,
outperforming the state-of-the-arts on various domain adaptation settings.
