  We propose two nonlinear regression methods, named Adversarial Orthogonal
Regression (AdOR) for additive noise models and Adversarial Orthogonal
Structural Equation Model (AdOSE) for the general case of structural equation
models. Both methods try to make the residual of regression independent from
regressors while putting no assumption on noise distribution. In both methods,
two adversarial networks are trained simultaneously where a regression network
outputs predictions and a loss network that estimates mutual information (in
AdOR) and KL-divergence (in AdOSE). These methods can be formulated as a
minimax two-player game; at equilibrium, AdOR finds a deterministic map between
inputs and output and estimates mutual information between residual and inputs,
while AdOSE estimates a conditional probability distribution of output given
inputs. The proposed methods can be used as subroutines to address several
learning problems in causality, such as causal direction determination (or more
generally, causal structure learning) and causal model estimation. Synthetic
and real-world experiments demonstrate that the proposed methods have a
remarkable performance with respect to previous solutions.
