  Recovering the 3D shape of transparent objects using a small number of
unconstrained natural images is an ill-posed problem. Complex light paths
induced by refraction and reflection have prevented both traditional and deep
multiview stereo from solving this challenge. We propose a physically-based
network to recover 3D shape of transparent objects using a few images acquired
with a mobile phone camera, under a known but arbitrary environment map. Our
novel contributions include a normal representation that enables the network to
model complex light transport through local computation, a rendering layer that
models refractions and reflections, a cost volume specifically designed for
normal refinement of transparent shapes and a feature mapping based on
predicted normals for 3D point cloud reconstruction. We render a synthetic
dataset to encourage the model to learn refractive light transport across
different views. Our experiments show successful recovery of high-quality 3D
geometry for complex transparent shapes using as few as 5-12 natural images.
Code and data are publicly released.
