  Traditional multi-view photometric stereo (MVPS) methods are often composed
of multiple disjoint stages, resulting in noticeable accumulated errors. In
this paper, we present a neural inverse rendering method for MVPS based on
implicit representation. Given multi-view images of a non-Lambertian object
illuminated by multiple unknown directional lights, our method jointly
estimates the geometry, materials, and lights. Our method first employs
multi-light images to estimate per-view surface normal maps, which are used to
regularize the normals derived from the neural radiance field. It then jointly
optimizes the surface normals, spatially-varying BRDFs, and lights based on a
shadow-aware differentiable rendering layer. After optimization, the
reconstructed object can be used for novel-view rendering, relighting, and
material editing. Experiments on both synthetic and real datasets demonstrate
that our method achieves far more accurate shape reconstruction than existing
MVPS and neural rendering methods. Our code and model can be found at
https://ywq.github.io/psnerf.
