  Knowledge of 3D properties of objects is a necessity in order to build
effective computer vision systems. However, lack of large scale 3D datasets can
be a major constraint for data-driven approaches in learning such properties.
We consider the task of single image 3D point cloud reconstruction, and aim to
utilize multiple foreground masks as our supervisory data to alleviate the need
for large scale 3D datasets. A novel differentiable projection module, called
'CAPNet', is introduced to obtain such 2D masks from a predicted 3D point
cloud. The key idea is to model the projections as a continuous approximation
of the points in the point cloud. To overcome the challenges of sparse
projection maps, we propose a loss formulation termed 'affinity loss' to
generate outlier-free reconstructions. We significantly outperform the existing
projection based approaches on a large-scale synthetic dataset. We show the
utility and generalizability of such a 2D supervised approach through
experiments on a real-world dataset, where lack of 3D data can be a serious
concern. To further enhance the reconstructions, we also propose a test stage
optimization procedure to obtain reconstructions that display high
correspondence with the observed input image.
