  We present Blocks2World, a novel method for 3D scene rendering and editing
that leverages a two-step process: convex decomposition of images and
conditioned synthesis. Our technique begins by extracting 3D parallelepipeds
from various objects in a given scene using convex decomposition, thus
obtaining a primitive representation of the scene. These primitives are then
utilized to generate paired data through simple ray-traced depth maps. The next
stage involves training a conditioned model that learns to generate images from
the 2D-rendered convex primitives. This step establishes a direct mapping
between the 3D model and its 2D representation, effectively learning the
transition from a 3D model to an image. Once the model is fully trained, it
offers remarkable control over the synthesis of novel and edited scenes. This
is achieved by manipulating the primitives at test time, including translating
or adding them, thereby enabling a highly customizable scene rendering process.
Our method provides a fresh perspective on 3D scene rendering and editing,
offering control and flexibility. It opens up new avenues for research and
applications in the field, including authoring and data augmentation.
