  In this paper, we tackle the domain adaptive object detection problem, where
the main challenge lies in significant domain gaps between source and target
domains. Previous work seeks to plainly align image-level and instance-level
shifts to eventually minimize the domain discrepancy. However, they still
overlook to match crucial image regions and important instances across domains,
which will strongly affect domain shift mitigation. In this work, we propose a
simple but effective categorical regularization framework for alleviating this
issue. It can be applied as a plug-and-play component on a series of Domain
Adaptive Faster R-CNN methods which are prominent for dealing with domain
adaptive detection. Specifically, by integrating an image-level multi-label
classifier upon the detection backbone, we can obtain the sparse but crucial
image regions corresponding to categorical information, thanks to the weakly
localization ability of the classification manner. Meanwhile, at the instance
level, we leverage the categorical consistency between image-level predictions
(by the classifier) and instance-level predictions (by the detection head) as a
regularization factor to automatically hunt for the hard aligned instances of
target domains. Extensive experiments of various domain shift scenarios show
that our method obtains a significant performance gain over original Domain
Adaptive Faster R-CNN detectors. Furthermore, qualitative visualization and
analyses can demonstrate the ability of our method for attending on the key
regions/instances targeting on domain adaptation. Our code is open-source and
available at \url{https://github.com/Megvii-Nanjing/CR-DA-DET}.
