  Domain adaptation methods face performance degradation in object detection,
as the complexity of tasks require more about the transferability of the model.
We propose a new perspective on how CNN models gain the transferability,
viewing the weights of a model as a series of motion patterns. The directions
of weights, and the gradients, can be divided into domain-specific and
domain-invariant parts, and the goal of domain adaptation is to concentrate on
the domain-invariant direction while eliminating the disturbance from
domain-specific one. Current UDA object detection methods view the two
directions as a whole while optimizing, which will cause domain-invariant
direction mismatch even if the output features are perfectly aligned. In this
paper, we propose the domain-specific suppression, an exemplary and
generalizable constraint to the original convolution gradients in
backpropagation to detach the two parts of directions and suppress the
domain-specific one. We further validate our theoretical analysis and methods
on several domain adaptive object detection tasks, including weather, camera
configuration, and synthetic to real-world adaptation. Our experiment results
show significant advance over the state-of-the-art methods in the UDA object
detection field, performing a promotion of $10.2\sim12.2\%$ mAP on all these
domain adaptation scenarios.
