  We propose GroundUp, the first sketch-based ideation tool for 3D city massing
of urban areas. We focus on early-stage urban design, where sketching is a
common tool and the design starts from balancing building volumes (masses) and
open spaces. With Human-Centered AI in mind, we aim to help architects quickly
revise their ideas by easily switching between 2D sketches and 3D models,
allowing for smoother iteration and sharing of ideas. Inspired by feedback from
architects and existing workflows, our system takes as a first input a user
sketch of multiple buildings in a top-down view. The user then draws a
perspective sketch of the envisioned site. Our method is designed to exploit
the complementarity of information in the two sketches and allows users to
quickly preview and adjust the inferred 3D shapes. Our model has two main
components. First, we propose a novel sketch-to-depth prediction network for
perspective sketches that exploits top-down sketch shapes. Second, we use depth
cues derived from the perspective sketch as a condition to our diffusion model,
which ultimately completes the geometry in a top-down view. Thus, our final 3D
geometry is represented as a heightfield, allowing users to construct the city
`from the ground up'.
