  The integration of molecule and language has garnered increasing attention in
molecular science. Recent advancements in Language Models (LMs) have
demonstrated potential for the comprehensive modeling of molecule and language.
However, existing works exhibit notable limitations. Most existing works
overlook the modeling of 3D information, which is crucial for understanding
molecular structures and also functions. While some attempts have been made to
leverage external structure encoding modules to inject the 3D molecular
information into LMs, there exist obvious difficulties that hinder the
integration of molecular structure and language text, such as modality
alignment and separate tuning. To bridge this gap, we propose 3D-MolT5, a
unified framework designed to model both 1D molecular sequence and 3D molecular
structure. The key innovation lies in our methodology for mapping fine-grained
3D substructure representations (based on 3D molecular fingerprints) to a
specialized 3D token vocabulary for 3D-MolT5. This 3D structure token
vocabulary enables the seamless combination of 1D sequence and 3D structure
representations in a tokenized format, allowing 3D-MolT5 to encode molecular
sequence (SELFIES), molecular structure, and text sequences within a unified
architecture. Alongside, we further introduce 1D and 3D joint pre-training to
enhance the model's comprehension of these diverse modalities in a joint
representation space and better generalize to various tasks for our foundation
model. Through instruction tuning on multiple downstream datasets, our proposed
3D-MolT5 shows superior performance than existing methods in molecular property
prediction, molecule captioning, and text-based molecule generation tasks. Our
code will be available on GitHub soon.
