  Fair and unbiased machine learning is an important and active field of
research, as decision processes are increasingly driven by models that learn
from data. Unfortunately, any biases present in the data may be learned by the
model, thereby inappropriately transferring that bias into the decision making
process. We identify the connection between the task of bias reduction and that
of isolating factors common between domains whilst encouraging domain specific
invariance. To isolate the common factors we combine the theory of deep latent
variable models with information bottleneck theory for scenarios whereby data
may be naturally paired across domains and no additional supervision is
required. The result is the Nested Variational AutoEncoder (NestedVAE). Two
outer VAEs with shared weights attempt to reconstruct the input and infer a
latent space, whilst a nested VAE attempts to reconstruct the latent
representation of one image, from the latent representation of its paired
image. In so doing, the nested VAE isolates the common latent factors/causes
and becomes invariant to unwanted factors that are not shared between paired
images. We also propose a new metric to provide a balanced method of evaluating
consistency and classifier performance across domains which we refer to as the
Adjusted Parity metric. An evaluation of NestedVAE on both domain and attribute
invariance, change detection, and learning common factors for the prediction of
biological sex demonstrates that NestedVAE significantly outperforms
alternative methods.
