  Shape from Polarization (SfP) estimates surface normals using photos captured
at different polarizer rotations. Fundamentally, the SfP model assumes that
light is reflected either diffusely or specularly. However, this model is not
valid for many real-world surfaces exhibiting a mixture of diffuse and specular
properties. To address this challenge, previous methods have used a sequential
solution: first, use an existing algorithm to separate the scene into diffuse
and specular components, then apply the appropriate SfP model. In this paper,
we propose a new method that jointly uses viewpoint and polarization data to
holistically separate diffuse and specular components, recover refractive
index, and ultimately recover 3D shape. By involving the physics of
polarization in the separation process, we demonstrate competitive results with
a benchmark method, while recovering additional information (e.g. refractive
index).
