  This paper addresses the problem of photometric stereo, in both calibrated
and uncalibrated scenarios, for non-Lambertian surfaces based on deep learning.
We first introduce a fully convolutional deep network for calibrated
photometric stereo, which we call PS-FCN. Unlike traditional approaches that
adopt simplified reflectance models to make the problem tractable, our method
directly learns the mapping from reflectance observations to surface normal,
and is able to handle surfaces with general and unknown isotropic reflectance.
At test time, PS-FCN takes an arbitrary number of images and their associated
light directions as input and predicts a surface normal map of the scene in a
fast feed-forward pass. To deal with the uncalibrated scenario where light
directions are unknown, we introduce a new convolutional network, named LCNet,
to estimate light directions from input images. The estimated light directions
and the input images are then fed to PS-FCN to determine the surface normals.
Our method does not require a pre-defined set of light directions and can
handle multiple images in an order-agnostic manner. Thorough evaluation of our
approach on both synthetic and real datasets shows that it outperforms
state-of-the-art methods in both calibrated and uncalibrated scenarios.
