  We tackle the task of learning dynamic 3D semantic radiance fields given a
single monocular video as input. Our learned semantic radiance field captures
per-point semantics as well as color and geometric properties for a dynamic 3D
scene, enabling the generation of novel views and their corresponding
semantics. This enables the segmentation and tracking of a diverse set of 3D
semantic entities, specified using a simple and intuitive interface that
includes a user click or a text prompt. To this end, we present DGD, a unified
3D representation for both the appearance and semantics of a dynamic 3D scene,
building upon the recently proposed dynamic 3D Gaussians representation. Our
representation is optimized over time with both color and semantic information.
Key to our method is the joint optimization of the appearance and semantic
attributes, which jointly affect the geometric properties of the scene. We
evaluate our approach in its ability to enable dense semantic 3D object
tracking and demonstrate high-quality results that are fast to render, for a
diverse set of scenes. Our project webpage is available on
https://isaaclabe.github.io/DGD-Website/
