  We present a novel convolutional neural network architecture for photometric
stereo (Woodham, 1980), a problem of recovering 3D object surface normals from
multiple images observed under varying illuminations. Despite its long history
in computer vision, the problem still shows fundamental challenges for surfaces
with unknown general reflectance properties (BRDFs). Leveraging deep neural
networks to learn complicated reflectance models is promising, but studies in
this direction are very limited due to difficulties in acquiring accurate
ground truth for training and also in designing networks invariant to
permutation of input images. In order to address these challenges, we propose a
physics based unsupervised learning framework where surface normals and BRDFs
are predicted by the network and fed into the rendering equation to synthesize
observed images. The network weights are optimized during testing by minimizing
reconstruction loss between observed and synthesized images. Thus, our learning
process does not require ground truth normals or even pre-training on external
images. Our method is shown to achieve the state-of-the-art performance on a
challenging real-world scene benchmark.
