  The tradeoff between precision and performance in molecular simulations can
nowadays be addressed by machine-learned force fields (MLFF), which combine
\textit{ab initio} accuracy with force field numerical efficiency. Different
from conventional force fields however, incorporating relevant electronic
degrees of freedom into MLFFs becomes important. Here, we implement an
equivariant transformer that embeds molecular net charge and spin state without
additional neural network parameters. The model trained on a singlet/triplet
non-correlated \ce{CH2} dataset can identify different spin states and shows
state-of-the-art extrapolation capability. Therein, self-attention sensibly
captures non-local effects, which, as we show, can be finely tuned over the
network hyper-parameters. We indeed found that Softmax activation functions
utilised in the self-attention mechanism of graph networks outperformed
ReLU-like functions in prediction accuracy. Increasing the attention
temperature from $\tau = \sqrt{d}$ to $\sqrt{2d}$ further improved the
extrapolation capability, indicating a weighty role of nonlocality.
Additionally, a weight initialisation method was purposed that sensibly
accelerated the training process.
