  Despite the significant success of deep learning in computer vision tasks,
cross-domain tasks still present a challenge in which the model's performance
will degrade when the training set and the test set follow different
distributions. Most existing methods employ adversarial learning or instance
normalization for achieving data augmentation to solve this task. In contrast,
considering that the batch normalization (BN) layer may not be robust for
unseen domains and there exist the differences between local patches of an
image, we propose a novel method called patch-aware batch normalization (PBN).
To be specific, we first split feature maps of a batch into non-overlapping
patches along the spatial dimension, and then independently normalize each
patch to jointly optimize the shared BN parameter at each iteration. By
exploiting the differences between local patches of an image, our proposed PBN
can effectively enhance the robustness of the model's parameters. Besides,
considering the statistics from each patch may be inaccurate due to their
smaller size compared to the global feature maps, we incorporate the globally
accumulated statistics with the statistics from each batch to obtain the final
statistics for normalizing each patch. Since the proposed PBN can replace the
typical BN, it can be integrated into most existing state-of-the-art methods.
Extensive experiments and analysis demonstrate the effectiveness of our PBN in
multiple computer vision tasks, including classification, object detection,
instance retrieval, and semantic segmentation.
