  Optimization of quantum states using the variational principle has recently
seen an upsurge due to developments of increasingly expressive wave functions.
In order to improve on the accuracy of the ans\"atze, it is a time-honored
strategy to impose the systems' symmetries. We present an ansatz where global
non-abelian symmetries are inherently embedded in its structure. We extend the
model to incorporate lattice symmetries as well. We consider the prototypical
example of the frustrated two-dimensional $J_1$-$J_2$ model on a square
lattice, for which eigenstates have been hard to model variationally. Our novel
approach guarantees that the obtained ground state will have total spin zero.
Benchmarks on the 2D $J_1$-$J_2$ model demonstrate its state-of-the-art
performance in representing the ground state. Furthermore, our methodology
permits to find the wave functions of excited states with definite quantum
numbers associated to the considered symmetries (including the non-abelian
ones), without modifying the architecture of the network.
