  We consider the problem of editing 3D objects and scenes based on open-ended
language instructions. A common approach to this problem is to use a 2D image
generator or editor to guide the 3D editing process, obviating the need for 3D
data. However, this process is often inefficient due to the need for iterative
updates of costly 3D representations, such as neural radiance fields, either
through individual view edits or score distillation sampling. A major
disadvantage of this approach is the slow convergence caused by aggregating
inconsistent information across views, as the guidance from 2D models is not
multi-view consistent. We thus introduce the Direct Gaussian Editor (DGE), a
method that addresses these issues in two stages. First, we modify a given
high-quality image editor like InstructPix2Pix to be multi-view consistent. To
do so, we propose a training-free approach that integrates cues from the 3D
geometry of the underlying scene. Second, given a multi-view consistent edited
sequence of images, we directly and efficiently optimize the 3D representation,
which is based on 3D Gaussian Splatting. Because it avoids incremental and
iterative edits, DGE is significantly more accurate and efficient than existing
approaches and offers additional benefits, such as enabling selective editing
of parts of the scene.
