  Although LEGO sets have entertained generations of children and adults, the
challenge of designing customized builds matching the complexity of real-world
or imagined scenes remains too great for the average enthusiast. In order to
make this feat possible, we implement a system that generates a LEGO brick
model from 2D images. We design a novel solution to this problem that uses an
octree-structured autoencoder trained on 3D voxelized models to obtain a
feasible latent representation for model reconstruction, and a separate network
trained to predict this latent representation from 2D images. LEGO models are
obtained by algorithmic conversion of the 3D voxelized model to bricks. We
demonstrate first-of-its-kind conversion of photographs to 3D LEGO models. An
octree architecture enables the flexibility to produce multiple resolutions to
best fit a user's creative vision or design needs. In order to demonstrate the
broad applicability of our system, we generate step-by-step building
instructions and animations for LEGO models of objects and human faces.
Finally, we test these automatically generated LEGO sets by constructing
physical builds using real LEGO bricks.
