  Learning to generate 3D point clouds without 3D supervision is an important
but challenging problem. Current solutions leverage various differentiable
renderers to project the generated 3D point clouds onto a 2D image plane, and
train deep neural networks using the per-pixel difference with 2D ground truth
images. However, these solutions are still struggling to fully recover fine
structures of 3D shapes, such as thin tubes or planes. To resolve this issue,
we propose an unsupervised approach for 3D point cloud generation with fine
structures. Specifically, we cast 3D point cloud learning as a 2D projection
matching problem. Rather than using entire 2D silhouette images as a regular
pixel supervision, we introduce structure adaptive sampling to randomly sample
2D points within the silhouettes as an irregular point supervision, which
alleviates the consistency issue of sampling from different view angles. Our
method pushes the neural network to generate a 3D point cloud whose 2D
projections match the irregular point supervision from different view angles.
Our 2D projection matching approach enables the neural network to learn more
accurate structure information than using the per-pixel difference, especially
for fine and thin 3D structures. Our method can recover fine 3D structures from
2D silhouette images at different resolutions, and is robust to different
sampling methods and point number in irregular point supervision. Our method
outperforms others under widely used benchmarks. Our code, data and models are
available at https://github.com/chenchao15/2D\_projection\_matching.
