  Given multiple labeled source domains and a single target domain, most
existing multi-source domain adaptation (MSDA) models are trained on data from
all domains jointly in one step. Such an one-step approach limits their ability
to adapt to the target domain. This is because the training set is dominated by
the more numerous and labeled source domain data. The source-domain-bias can
potentially be alleviated by introducing a second training step, where the
model is fine-tuned with the unlabeled target domain data only using pseudo
labels as supervision. However, the pseudo labels are inevitably noisy and when
used unchecked can negatively impact the model performance. To address this
problem, we propose a novel Bi-level Optimization based Robust Target Training
(BORT$^2$) method for MSDA. Given any existing fully-trained one-step MSDA
model, BORT$^2$ turns it to a labeling function to generate pseudo-labels for
the target data and trains a target model using pseudo-labeled target data
only. Crucially, the target model is a stochastic CNN which is designed to be
intrinsically robust against label noise generated by the labeling function.
Such a stochastic CNN models each target instance feature as a Gaussian
distribution with an entropy maximization regularizer deployed to measure the
label uncertainty, which is further exploited to alleviate the negative impact
of noisy pseudo labels. Training the labeling function and the target model
poses a nested bi-level optimization problem, for which we formulate an elegant
solution based on implicit differentiation. Extensive experiments demonstrate
that our proposed method achieves the state of the art performance on three
MSDA benchmarks, including the large-scale DomainNet dataset. Our code will be
available at \url{https://github.com/Zhongying-Deng/BORT2}
