  Distribution matching can be used to learn invariant representations with
applications in fairness and robustness. Most prior works resort to adversarial
matching methods but the resulting minimax problems are unstable and
challenging to optimize. Non-adversarial likelihood-based approaches either
require model invertibility, impose constraints on the latent prior, or lack a
generic framework for distribution matching. To overcome these limitations, we
propose a non-adversarial VAE-based matching method that can be applied to any
model pipeline. We develop a set of alignment upper bounds for distribution
matching (including a noisy bound) that have VAE-like objectives but with a
different perspective. We carefully compare our method to prior VAE-based
matching approaches both theoretically and empirically. Finally, we demonstrate
that our novel matching losses can replace adversarial losses in standard
invariant representation learning pipelines without modifying the original
architectures -- thereby significantly broadening the applicability of
non-adversarial matching methods.
