  Controllable scene synthesis consists of generating 3D information that
satisfy underlying specifications. Thereby, these specifications should be
abstract, i.e. allowing easy user interaction, whilst providing enough
interface for detailed control. Scene graphs are representations of a scene,
composed of objects (nodes) and inter-object relationships (edges), proven to
be particularly suited for this task, as they allow for semantic control on the
generated content. Previous works tackling this task often rely on synthetic
data, and retrieve object meshes, which naturally limits the generation
capabilities. To circumvent this issue, we instead propose the first work that
directly generates shapes from a scene graph in an end-to-end manner. In
addition, we show that the same model supports scene modification, using the
respective scene graph as interface. Leveraging Graph Convolutional Networks
(GCN) we train a variational Auto-Encoder on top of the object and edge
categories, as well as 3D shapes and scene layouts, allowing latter sampling of
new scenes and shapes.
