  Assuming the source label space subsumes the target one, Partial Video Domain
Adaptation (PVDA) is a more general and practical scenario for cross-domain
video classification problems. The key challenge of PVDA is to mitigate the
negative transfer caused by the source-only outlier classes. To tackle this
challenge, a crucial step is to aggregate target predictions to assign class
weights by up-weighing target classes and down-weighing outlier classes.
However, the incorrect predictions of class weights can mislead the network and
lead to negative transfer. Previous works improve the class weight accuracy by
utilizing temporal features and attention mechanisms, but these methods may
fall short when trying to generate accurate class weight when domain shifts are
significant, as in most real-world scenarios. To deal with these challenges, we
propose the Multi-modality Cluster-calibrated partial Adversarial Network
(MCAN). MCAN enhances video feature extraction with multi-modal features from
multiple temporal scales to form more robust overall features. It utilizes a
novel class weight calibration method to alleviate the negative transfer caused
by incorrect class weights. The calibration method tries to identify and weigh
correct and incorrect predictions using distributional information implied by
unsupervised clustering. Extensive experiments are conducted on prevailing PVDA
benchmarks, and the proposed MCAN achieves significant improvements when
compared to state-of-the-art PVDA methods.
