  Collider bias is a harmful form of sample selection bias that neural networks
are ill-equipped to handle. This bias manifests itself when the underlying
causal signal is strongly correlated with other confounding signals due to the
training data collection procedure. In the situation where the confounding
signal is easy-to-learn, deep neural networks will latch onto this and the
resulting model will generalise poorly to in-the-wild test scenarios. We argue
herein that the cause of failure is a combination of the deep structure of
neural networks and the greedy gradient-driven learning process used - one that
prefers easy-to-compute signals when available. We show it is possible to
mitigate against this by generating bias-decoupled training data using latent
adversarial debiasing (LAD), even when the confounding signal is present in
100% of the training data. By training neural networks on these adversarial
examples,we can improve their generalisation in collider bias settings.
Experiments show state-of-the-art performance of LAD in label-free debiasing
with gains of 76.12% on background coloured MNIST, 35.47% on fore-ground
coloured MNIST, and 8.27% on corrupted CIFAR-10.
