  3D reconstruction and simulation, while interrelated, have distinct
objectives: reconstruction demands a flexible 3D representation adaptable to
diverse scenes, whereas simulation requires a structured representation to
model motion principles effectively. This paper introduces the Mesh-adsorbed
Gaussian Splatting (MaGS) method to resolve such a dilemma. MaGS constrains 3D
Gaussians to hover on the mesh surface, creating a mutual-adsorbed
mesh-Gaussian 3D representation that combines the rendering flexibility of 3D
Gaussians with the spatial coherence of meshes. Leveraging this representation,
we introduce a learnable Relative Deformation Field (RDF) to model the relative
displacement between the mesh and 3D Gaussians, extending traditional
mesh-driven deformation paradigms that only rely on ARAP prior, thus capturing
the motion of each 3D Gaussian more precisely. By joint optimizing meshes, 3D
Gaussians, and RDF, MaGS achieves both high rendering accuracy and realistic
deformation. Extensive experiments on the D-NeRF and NeRF-DS datasets
demonstrate that MaGS can generate competitive results in both reconstruction
and simulation.
