  Photorealistic 3D reconstruction of street scenes is a critical technique for
developing real-world simulators for autonomous driving. Despite the efficacy
of Neural Radiance Fields (NeRF) for driving scenes, 3D Gaussian Splatting
(3DGS) emerges as a promising direction due to its faster speed and more
explicit representation. However, most existing street 3DGS methods require
tracked 3D vehicle bounding boxes to decompose the static and dynamic elements
for effective reconstruction, limiting their applications for in-the-wild
scenarios. To facilitate efficient 3D scene reconstruction without costly
annotations, we propose a self-supervised street Gaussian
($\textit{S}^3$Gaussian) method to decompose dynamic and static elements from
4D consistency. We represent each scene with 3D Gaussians to preserve the
explicitness and further accompany them with a spatial-temporal field network
to compactly model the 4D dynamics. We conduct extensive experiments on the
challenging Waymo-Open dataset to evaluate the effectiveness of our method. Our
$\textit{S}^3$Gaussian demonstrates the ability to decompose static and dynamic
scenes and achieves the best performance without using 3D annotations. Code is
available at: https://github.com/nnanhuang/S3Gaussian/.
