  Photometric stereo provides an important method for high-fidelity 3D
reconstruction based on multiple intensity images captured under different
illumination directions. In this paper, we present a complete framework,
including a multilight source illumination and acquisition hardware system and
a two-stage convolutional neural network (CNN) architecture, to construct
inter- and intraframe representations for accurate normal estimation of
non-Lambertian objects. We experimentally investigate numerous network design
alternatives for identifying the optimal scheme to deploy inter- and intraframe
feature extraction modules for the photometric stereo problem. Moreover, we
propose utilizing the easily obtained object mask to eliminate adverse
interference from invalid background regions in intraframe spatial
convolutions, thus effectively improving the accuracy of normal estimation for
surfaces made of dark materials or with cast shadows. Experimental results
demonstrate that the proposed masked two-stage photometric stereo CNN model
(MT-PS-CNN) performs favourably against state-of-the-art photometric stereo
techniques in terms of both accuracy and efficiency. In addition, the proposed
method is capable of predicting accurate and rich surface normal details for
non-Lambertian objects of complex geometry and performs stably given inputs
captured in both sparse and dense lighting distributions.
