  Learning meaningful representations of data that can address challenges such
as batch effect correction and counterfactual inference is a central problem in
many domains including computational biology. Adopting a Conditional VAE
framework, we show that marginal independence between the representation and a
condition variable plays a key role in both of these challenges. We propose the
Contrastive Mixture of Posteriors (CoMP) method that uses a novel misalignment
penalty defined in terms of mixtures of the variational posteriors to enforce
this independence in latent space. We show that CoMP has attractive theoretical
properties compared to previous approaches, and we prove counterfactual
identifiability of CoMP under additional assumptions. We demonstrate
state-of-the-art performance on a set of challenging tasks including aligning
human tumour samples with cancer cell-lines, predicting transcriptome-level
perturbation responses, and batch correction on single-cell RNA sequencing
data. We also find parallels to fair representation learning and demonstrate
that CoMP is competitive on a common task in the field.
