  We study the problem of choosing algorithm hyper-parameters in unsupervised
domain adaptation, i.e., with labeled data in a source domain and unlabeled
data in a target domain, drawn from a different input distribution. We follow
the strategy to compute several models using different hyper-parameters, and,
to subsequently compute a linear aggregation of the models. While several
heuristics exist that follow this strategy, methods are still missing that rely
on thorough theories for bounding the target error. In this turn, we propose a
method that extends weighted least squares to vector-valued functions, e.g.,
deep neural networks. We show that the target error of the proposed algorithm
is asymptotically not worse than twice the error of the unknown optimal
aggregation. We also perform a large scale empirical comparative study on
several datasets, including text, images, electroencephalogram, body sensor
signals and signals from mobile phones. Our method outperforms deep embedded
validation (DEV) and importance weighted validation (IWV) on all datasets,
setting a new state-of-the-art performance for solving parameter choice issues
in unsupervised domain adaptation with theoretical error guarantees. We further
study several competitive heuristics, all outperforming IWV and DEV on at least
five datasets. However, our method outperforms each heuristic on at least five
of seven datasets.
