  We propose GS-IR, a novel inverse rendering approach based on 3D Gaussian
Splatting (GS) that leverages forward mapping volume rendering to achieve
photorealistic novel view synthesis and relighting results. Unlike previous
works that use implicit neural representations and volume rendering (e.g.
NeRF), which suffer from low expressive power and high computational
complexity, we extend GS, a top-performance representation for novel view
synthesis, to estimate scene geometry, surface material, and environment
illumination from multi-view images captured under unknown lighting conditions.
There are two main problems when introducing GS to inverse rendering: 1) GS
does not support producing plausible normal natively; 2) forward mapping (e.g.
rasterization and splatting) cannot trace the occlusion like backward mapping
(e.g. ray tracing). To address these challenges, our GS-IR proposes an
efficient optimization scheme that incorporates a depth-derivation-based
regularization for normal estimation and a baking-based occlusion to model
indirect lighting. The flexible and expressive GS representation allows us to
achieve fast and compact geometry reconstruction, photorealistic novel view
synthesis, and effective physically-based rendering. We demonstrate the
superiority of our method over baseline methods through qualitative and
quantitative evaluations on various challenging scenes.
