  Molecular representation pretraining is critical in various applications for
drug and material discovery due to the limited number of labeled molecules, and
most existing work focuses on pretraining on 2D molecular graphs. However, the
power of pretraining on 3D geometric structures has been less explored. This is
owing to the difficulty of finding a sufficient proxy task that can empower the
pretraining to effectively extract essential features from the geometric
structures. Motivated by the dynamic nature of 3D molecules, where the
continuous motion of a molecule in the 3D Euclidean space forms a smooth
potential energy surface, we propose GeoSSL, a 3D coordinate denoising
pretraining framework to model such an energy landscape. Further by leveraging
an SE(3)-invariant score matching method, we propose GeoSSL-DDM in which the
coordinate denoising proxy task is effectively boiled down to denoising the
pairwise atomic distances in a molecule. Our comprehensive experiments confirm
the effectiveness and robustness of our proposed method.
