  Realistic conditional 3D scene synthesis significantly enhances and
accelerates the creation of virtual environments, which can also provide
extensive training data for computer vision and robotics research among other
applications. Diffusion models have shown great performance in related
applications, e.g., making precise arrangements of unordered sets. However,
these models have not been fully explored in floor-conditioned scene synthesis
problems. We present MiDiffusion, a novel mixed discrete-continuous diffusion
model architecture, designed to synthesize plausible 3D indoor scenes from
given room types, floor plans, and potentially pre-existing objects. We
represent a scene layout by a 2D floor plan and a set of objects, each defined
by its category, location, size, and orientation. Our approach uniquely
implements structured corruption across the mixed discrete semantic and
continuous geometric domains, resulting in a better conditioned problem for the
reverse denoising step. We evaluate our approach on the 3D-FRONT dataset. Our
experimental results demonstrate that MiDiffusion substantially outperforms
state-of-the-art autoregressive and diffusion models in floor-conditioned 3D
scene synthesis. In addition, our models can handle partial object constraints
via a corruption-and-masking strategy without task specific training. We show
MiDiffusion maintains clear advantages over existing approaches in scene
completion and furniture arrangement experiments.
