  Photometric stereo leverages variations in illumination conditions to
reconstruct surface normals. Display photometric stereo, which employs a
conventional monitor as an illumination source, has the potential to overcome
limitations often encountered in bulky and difficult-to-use conventional
setups. In this paper, we present differentiable display photometric stereo
(DDPS), addressing an often overlooked challenge in display photometric stereo:
the design of display patterns. Departing from using heuristic display
patterns, DDPS learns the display patterns that yield accurate normal
reconstruction for a target system in an end-to-end manner. To this end, we
propose a differentiable framework that couples basis-illumination image
formation with analytic photometric-stereo reconstruction. The differentiable
framework facilitates the effective learning of display patterns via
auto-differentiation. Also, for training supervision, we propose to use 3D
printing for creating a real-world training dataset, enabling accurate
reconstruction on the target real-world setup. Finally, we exploit that
conventional LCD monitors emit polarized light, which allows for the optical
separation of diffuse and specular reflections when combined with a
polarization camera, leading to accurate normal reconstruction. Extensive
evaluation of DDPS shows improved normal-reconstruction accuracy compared to
heuristic patterns and demonstrates compelling properties such as robustness to
pattern initialization, calibration errors, and simplifications in image
formation and reconstruction.
