  In many important machine learning applications, the training distribution
used to learn a probabilistic classifier differs from the testing distribution
on which the classifier will be used to make predictions. Traditional methods
correct the distribution shift by reweighting the training data with the ratio
of the density between test and training data. In many applications training
takes place without prior knowledge of the testing distribution on which the
algorithm will be applied in the future. Recently, methods have been proposed
to address the shift by learning causal structure, but those methods rely on
the diversity of multiple training data to a good performance, and have
complexity limitations in high dimensions. In this paper, we propose a novel
Deep Global Balancing Regression (DGBR) algorithm to jointly optimize a deep
auto-encoder model for feature selection and a global balancing model for
stable prediction across unknown environments. The global balancing model
constructs balancing weights that facilitate estimating of partial effects of
features (holding fixed all other features), a problem that is challenging in
high dimensions, and thus helps to identify stable, causal relationships
between features and outcomes. The deep auto-encoder model is designed to
reduce the dimensionality of the feature space, thus making global balancing
easier. We show, both theoretically and with empirical experiments, that our
algorithm can make stable predictions across unknown environments. Our
experiments on both synthetic and real world datasets demonstrate that our DGBR
algorithm outperforms the state-of-the-art methods for stable prediction across
unknown environments.
