  AI for drug discovery has been a research hotspot in recent years, and
SMILES-based language models has been increasingly applied in drug molecular
design. However, no work has explored whether and how language models
understand the chemical spatial structure from 1D sequences. In this work, we
pre-train a transformer model on chemical language and fine-tune it toward drug
design objectives, and investigate the correspondence between high-frequency
SMILES substrings and molecular fragments. The results indicate that language
models can understand chemical structures from the perspective of molecular
fragments, and the structural knowledge learned through fine-tuning is
reflected in the high-frequency SMILES substrings generated by the model.
