  We propose generative multitask learning (GMTL), a simple and scalable
approach to causal representation learning for multitask learning. Our approach
makes a minor change to the conventional multitask inference objective, and
improves robustness to target shift. Since GMTL only modifies the inference
objective, it can be used with existing multitask learning methods without
requiring additional training. The improvement in robustness comes from
mitigating unobserved confounders that cause the targets, but not the input. We
refer to them as \emph{target-causing confounders}. These confounders induce
spurious dependencies between the input and targets. This poses a problem for
conventional multitask learning, due to its assumption that the targets are
conditionally independent given the input. GMTL mitigates target-causing
confounding at inference time, by removing the influence of the joint target
distribution, and predicting all targets jointly. This removes the spurious
dependencies between the input and targets, where the degree of removal is
adjustable via a single hyperparameter. This flexibility is useful for managing
the trade-off between in- and out-of-distribution generalization. Our results
on the Attributes of People and Taskonomy datasets reflect an improved
robustness to target shift across four multitask learning methods.
