  We describe a method to train a generative model with latent factors that are
(approximately) independent and localized. This means that perturbing the
latent variables affects only local regions of the synthesized image,
corresponding to objects. Unlike other unsupervised generative models, ours
enables object-centric manipulation, without requiring object-level
annotations, or any form of annotation for that matter. The key to our method
is the combination of spatial disentanglement, enforced by a Contextual
Information Separation loss, and perceptual cycle-consistency, enforced by a
loss that penalizes changes in the image partition in response to perturbations
of the latent factors. We test our method's ability to allow independent
control of spatial and semantic factors of variability on existing datasets and
also introduce two new ones that highlight the limitations of current methods.
