  Neural fields (NeRF) have emerged as a promising approach for representing
continuous 3D scenes. Nevertheless, the lack of semantic encoding in NeRFs
poses a significant challenge for scene decomposition. To address this
challenge, we present a single model, Multi-Modal Decomposition NeRF
(${M^2D}$NeRF), that is capable of both text-based and visual patch-based
edits. Specifically, we use multi-modal feature distillation to integrate
teacher features from pretrained visual and language models into 3D semantic
feature volumes, thereby facilitating consistent 3D editing. To enforce
consistency between the visual and language features in our 3D feature volumes,
we introduce a multi-modal similarity constraint. We also introduce a
patch-based joint contrastive loss that helps to encourage object-regions to
coalesce in the 3D feature space, resulting in more precise boundaries.
Experiments on various real-world scenes show superior performance in 3D scene
decomposition tasks compared to prior NeRF-based methods.
