  Domain adaptation (DA) aims to transfer knowledge from a label-rich and
related domain (source domain) to a label-scare domain (target domain).
Pseudo-labeling has recently been widely explored and used in DA. However, this
line of research is still confined to the inaccuracy of pseudo-labels. In this
paper, we reveal an interesting observation that the target samples belonging
to the classes with larger domain shift are easier to be misclassified compared
with the other classes. These classes are called hard class, which deteriorates
the performance of DA and restricts the applications of DA. We propose a novel
framework, called Hard Class Rectification Pseudo-labeling (HCRPL), to
alleviate the hard class problem from two aspects. First, as is difficult to
identify the target samples as hard class, we propose a simple yet effective
scheme, named Adaptive Prediction Calibration (APC), to calibrate the
predictions of the target samples according to the difficulty degree for each
class. Second, we further consider that the predictions of target samples
belonging to the hard class are vulnerable to perturbations. To prevent these
samples to be misclassified easily, we introduce Temporal-Ensembling (TE) and
Self-Ensembling (SE) to obtain consistent predictions. The proposed method is
evaluated in both unsupervised domain adaptation (UDA) and semi-supervised
domain adaptation (SSDA). The experimental results on several real-world
cross-domain benchmarks, including ImageCLEF, Office-31 and Office-Home,
substantiates the superiority of the proposed method.
