  Estimating individual and average treatment effects from observational data
is an important problem in many domains such as healthcare and e-commerce. In
this paper, we advocate balance regularization of multi-head neural network
architectures. Our work is motivated by representation learning techniques to
reduce differences between treated and untreated distributions that potentially
arise due to confounding factors. We further regularize the model by
encouraging it to predict control outcomes for individuals in the treatment
group that are similar to control outcomes in the control group. We empirically
study the bias-variance trade-off between different weightings of the
regularizers, as well as between inductive and transductive inference.
