  A default assumption in many machine learning scenarios is that the training
and test samples are drawn from the same probability distribution. However,
such an assumption is often violated in the real world due to non-stationarity
of the environment or bias in sample selection. In this work, we consider a
prevalent setting called covariate shift, where the input distribution differs
between the training and test stages while the conditional distribution of the
output given the input remains unchanged. Most of the existing methods for
covariate shift adaptation are two-step approaches, which first calculate the
importance weights and then conduct importance-weighted empirical risk
minimization. In this paper, we propose a novel one-step approach that jointly
learns the predictive model and the associated weights in one optimization by
minimizing an upper bound of the test risk. We theoretically analyze the
proposed method and provide a generalization error bound. We also empirically
demonstrate the effectiveness of the proposed method.
