  Pursuing causality from data is a fundamental problem in scientific
discovery, treatment intervention, and transfer learning. This paper introduces
a novel algorithmic method for addressing nonparametric invariance and
causality learning in regression models across multiple environments, where the
joint distribution of response variables and covariates varies, but the
conditional expectations of outcome given an unknown set of quasi-causal
variables are invariant. The challenge of finding such an unknown set of
quasi-causal or invariant variables is compounded by the presence of endogenous
variables that have heterogeneous effects across different environments,
including even one of them in the regression would make the estimation
inconsistent. The proposed Focused Adversial Invariant Regularization (FAIR)
framework utilizes an innovative minimax optimization approach that breaks down
the barriers, driving regression models toward prediction-invariant solutions
through adversarial testing. Leveraging the representation power of neural
networks, FAIR neural networks (FAIR-NN) are introduced for causality pursuit.
It is shown that FAIR-NN can find the invariant variables and quasi-causal
variables under a minimal identification condition and that the resulting
procedure is adaptive to low-dimensional composition structures in a
non-asymptotic analysis. Under a structural causal model, variables identified
by FAIR-NN represent pragmatic causality and provably align with exact causal
mechanisms under conditions of sufficient heterogeneity. Computationally,
FAIR-NN employs a novel Gumbel approximation with decreased temperature and
stochastic gradient descent ascent algorithm. The procedures are convincingly
demonstrated using simulated and real-data examples.
