  Standard supervised learning breaks down under data distribution shift.
However, the principle of independent causal mechanisms (ICM, Peters et al.
(2017)) can turn this weakness into an opportunity: one can take advantage of
distribution shift between different environments during training in order to
obtain more robust models. We propose a new gradient-based learning framework
whose objective function is derived from the ICM principle. We show
theoretically and experimentally that neural networks trained in this framework
focus on relations remaining invariant across environments and ignore unstable
ones. Moreover, we prove that the recovered stable relations correspond to the
true causal mechanisms under certain conditions. In both regression and
classification, the resulting models generalize well to unseen scenarios where
traditionally trained models fail.
