  In recent years, molecular representation learning has emerged as a key area
of focus in various chemical tasks. However, many existing models fail to fully
consider the geometric information of molecular structures, resulting in less
intuitive representations. Moreover, the widely used message-passing mechanism
is limited to provide the interpretation of experimental results from a
chemical perspective. To address these challenges, we introduce a novel
Transformer-based framework for molecular representation learning, named the
Geometry-aware Transformer (GeoT). GeoT learns molecular graph structures
through attention-based mechanisms specifically designed to offer reliable
interpretability, as well as molecular property prediction. Consequently, GeoT
can generate attention maps of interatomic relationships associated with
training objectives. In addition, GeoT demonstrates comparable performance to
MPNN-based models while achieving reduced computational complexity. Our
comprehensive experiments, including an empirical simulation, reveal that GeoT
effectively learns the chemical insights into molecular structures, bridging
the gap between artificial intelligence and molecular sciences.
