  In this work, we present a novel framework built to simplify 3D asset
generation for amateur users. To enable interactive generation, our method
supports a variety of input modalities that can be easily provided by a human,
including images, text, partially observed shapes and combinations of these,
further allowing to adjust the strength of each input. At the core of our
approach is an encoder-decoder, compressing 3D shapes into a compact latent
representation, upon which a diffusion model is learned. To enable a variety of
multi-modal inputs, we employ task-specific encoders with dropout followed by a
cross-attention mechanism. Due to its flexibility, our model naturally supports
a variety of tasks, outperforming prior works on shape completion, image-based
3D reconstruction, and text-to-3D. Most interestingly, our model can combine
all these tasks into one swiss-army-knife tool, enabling the user to perform
shape generation using incomplete shapes, images, and textual descriptions at
the same time, providing the relative weights for each input and facilitating
interactivity. Despite our approach being shape-only, we further show an
efficient method to texture the generated shape using large-scale text-to-image
models.
