  State-of-the-art 3D models, which excel in recognition tasks, typically
depend on large-scale datasets and well-defined category sets. Recent advances
in multi-modal pre-training have demonstrated potential in learning 3D
representations by aligning features from 3D shapes with their 2D RGB or depth
counterparts. However, these existing frameworks often rely solely on either
RGB or depth images, limiting their effectiveness in harnessing a comprehensive
range of multi-modal data for 3D applications. To tackle this challenge, we
present DR-Point, a tri-modal pre-training framework that learns a unified
representation of RGB images, depth images, and 3D point clouds by pre-training
with object triplets garnered from each modality. To address the scarcity of
such triplets, DR-Point employs differentiable rendering to obtain various
depth images. This approach not only augments the supply of depth images but
also enhances the accuracy of reconstructed point clouds, thereby promoting the
representative learning of the Transformer backbone. Subsequently, using a
limited number of synthetically generated triplets, DR-Point effectively learns
a 3D representation space that aligns seamlessly with the RGB-Depth image
space. Our extensive experiments demonstrate that DR-Point outperforms existing
self-supervised learning methods in a wide range of downstream tasks, including
3D object classification, part segmentation, point cloud completion, semantic
segmentation, and detection. Additionally, our ablation studies validate the
effectiveness of DR-Point in enhancing point cloud understanding.
