  Recent advances in neural rendering have shown great potential for
reconstructing scenes from multiview images. However, accurately representing
objects with glossy surfaces remains a challenge for existing methods. In this
work, we introduce ENVIDR, a rendering and modeling framework for high-quality
rendering and reconstruction of surfaces with challenging specular reflections.
To achieve this, we first propose a novel neural renderer with decomposed
rendering components to learn the interaction between surface and environment
lighting. This renderer is trained using existing physically based renderers
and is decoupled from actual scene representations. We then propose an
SDF-based neural surface model that leverages this learned neural renderer to
represent general scenes. Our model additionally synthesizes indirect
illuminations caused by inter-reflections from shiny surfaces by marching
surface-reflected rays. We demonstrate that our method outperforms state-of-art
methods on challenging shiny scenes, providing high-quality rendering of
specular reflections while also enabling material editing and scene relighting.
