  The Transformer architecture has become the state-of-art model for natural
language processing tasks and, more recently, also for computer vision tasks,
thus defining the Vision Transformer (ViT) architecture. The key feature is the
ability to describe long-range correlations among the elements of the input
sequences, through the so-called self-attention mechanism. Here, we propose an
adaptation of the ViT architecture with complex parameters to define a new
class of variational neural-network states for quantum many-body systems, the
ViT wave function. We apply this idea to the one-dimensional $J_1$-$J_2$
Heisenberg model, demonstrating that a relatively simple parametrization gets
excellent results for both gapped and gapless phases. In this case, excellent
accuracies are obtained by a relatively shallow architecture, with a single
layer of self-attention, thus largely simplifying the original architecture.
Still, the optimization of a deeper structure is possible and can be used for
more challenging models, most notably highly-frustrated systems in two
dimensions. The success of the ViT wave function relies on mixing both local
and global operations, thus enabling the study of large systems with high
accuracy.
