  We show how to extend traditional intrinsic image decompositions to
incorporate further layers above albedo and shading. It is hard to obtain data
to learn a multi-layer decomposition. Instead, we can learn to decompose an
image into layers that are "like this" by authoring generative models for each
layer using proxy examples that capture the Platonic ideal (Mondrian images for
albedo; rendered 3D primitives for shading; material swatches for shading
detail). Our method then generates image layers, one from each model, that
explain the image. Our approach rests on innovation in generative models for
images. We introduce a Convolutional Variational Auto Encoder (conv-VAE), a
novel VAE architecture that can reconstruct high fidelity images. The approach
is general, and does not require that layers admit a physical interpretation.
