  Humans naturally retain memories of permanent elements, while ephemeral
moments often slip through the cracks of memory. This selective retention is
crucial for robotic perception, localization, and mapping. To endow robots with
this capability, we introduce 3D Gaussian Mapping (3DGM), a self-supervised,
camera-only offline mapping framework grounded in 3D Gaussian Splatting. 3DGM
converts multitraverse RGB videos from the same region into a Gaussian-based
environmental map while concurrently performing 2D ephemeral object
segmentation. Our key observation is that the environment remains consistent
across traversals, while objects frequently change. This allows us to exploit
self-supervision from repeated traversals to achieve environment-object
decomposition. More specifically, 3DGM formulates multitraverse environmental
mapping as a robust differentiable rendering problem, treating pixels of the
environment and objects as inliers and outliers, respectively. Using robust
feature distillation, feature residuals mining, and robust optimization, 3DGM
jointly performs 2D segmentation and 3D mapping without human intervention. We
build the Mapverse benchmark, sourced from the Ithaca365 and nuPlan datasets,
to evaluate our method in unsupervised 2D segmentation, 3D reconstruction, and
neural rendering. Extensive results verify the effectiveness and potential of
our method for self-driving and robotics.
