  This paper investigates the problem of learning robust, generalizable
prediction models from a combination of multiple datasets and qualitative
assumptions about the underlying data-generating model. Part of the challenge
of learning robust models lies in the influence of unobserved confounders that
void many of the invariances and principles of minimum error presently used for
this problem. Our approach is to define a different invariance property of
causal solutions in the presence of unobserved confounders which, through a
relaxation of this invariance, can be connected with an explicit
distributionally robust optimization problem over a set of affine combination
of data distributions. Concretely, our objective takes the form of a standard
loss, plus a regularization term that encourages partial equality of error
derivatives with respect to model parameters. We demonstrate the empirical
performance of our approach on healthcare data from different modalities,
including image, speech and tabular data.
