  Multidomain crowd counting aims to learn a general model for multiple diverse
datasets. However, deep networks prefer modeling distributions of the dominant
domains instead of all domains, which is known as domain bias. In this study,
we propose a simple-yet-effective Modulating Domain-specific Knowledge Network
(MDKNet) to handle the domain bias issue in multidomain crowd counting. MDKNet
is achieved by employing the idea of `modulating', enabling deep network
balancing and modeling different distributions of diverse datasets with little
bias. Specifically, we propose an Instance-specific Batch Normalization (IsBN)
module, which serves as a base modulator to refine the information flow to be
adaptive to domain distributions. To precisely modulating the domain-specific
information, the Domain-guided Virtual Classifier (DVC) is then introduced to
learn a domain-separable latent space. This space is employed as an input
guidance for the IsBN modulator, such that the mixture distributions of
multiple datasets can be well treated. Extensive experiments performed on
popular benchmarks, including Shanghai-tech A/B, QNRF and NWPU, validate the
superiority of MDKNet in tackling multidomain crowd counting and the
effectiveness for multidomain learning. Code is available at
\url{https://github.com/csguomy/MDKNet}.
