  Due to their inherent capabilities of capturing non-local dependencies,
Transformer neural networks have quickly been established as the paradigmatic
architecture for large language models and image processing. Next to these
traditional applications, machine learning methods have also been demonstrated
to be versatile tools in the analysis of image-like data of quantum phases of
matter, e.g. given snapshots of many-body wave functions obtained in ultracold
atom experiments. While local correlation structures in image-like data of
physical systems can reliably be detected, identifying phases of matter
characterized by global, non-local structures with interpretable machine
learning methods remains a challenge. Here, we introduce the correlator
Transformer (CoTra), which classifies different phases of matter while at the
same time yielding full interpretability in terms of physical correlation
functions. The network's underlying structure is a tailored attention
mechanism, which learns efficient ways to weigh local and non-local
correlations for a successful classification. We demonstrate the versatility of
the CoTra by detecting local order in the Heisenberg antiferromagnet, and show
that local gauge constraints in one- and two-dimensional lattice gauge theories
can be identified. Furthermore, we establish that the CoTra reliably detects
non-local structures in images of correlated fermions in momentum space (Cooper
pairs) and that it can distinguish percolating from non-percolating images.
