  Identifying meaningful and independent factors of variation in a dataset is a
challenging learning task frequently addressed by means of deep latent variable
models. This task can be viewed as learning symmetry transformations preserving
the value of a chosen property along latent dimensions. However, existing
approaches exhibit severe drawbacks in enforcing the invariance property in the
latent space. We address these shortcomings with a novel approach to cycle
consistency. Our method involves two separate latent subspaces for the target
property and the remaining input information, respectively. In order to enforce
invariance as well as sparsity in the latent space, we incorporate semantic
knowledge by using cycle consistency constraints relying on property side
information. The proposed method is based on the deep information bottleneck
and, in contrast to other approaches, allows using continuous target properties
and provides inherent model selection capabilities. We demonstrate on synthetic
and molecular data that our approach identifies more meaningful factors which
lead to sparser and more interpretable models with improved invariance
properties.
