  Deep neural networks can model images with rich latent representations, but
they cannot naturally conceptualize structures of object categories in a
human-perceptible way. This paper addresses the problem of learning object
structures in an image modeling process without supervision. We propose an
autoencoding formulation to discover landmarks as explicit structural
representations. The encoding module outputs landmark coordinates, whose
validity is ensured by constraints that reflect the necessary properties for
landmarks. The decoding module takes the landmarks as a part of the learnable
input representations in an end-to-end differentiable framework. Our discovered
landmarks are semantically meaningful and more predictive of manually annotated
landmarks than those discovered by previous methods. The coordinates of our
landmarks are also complementary features to pretrained deep-neural-network
representations in recognizing visual attributes. In addition, the proposed
method naturally creates an unsupervised, perceptible interface to manipulate
object shapes and decode images with controllable structures. The project
webpage is at http://ytzhang.net/projects/lmdis-rep
