  We investigate the ground state phase diagram of an SU($N$)-symmetric
antiferromagnetic spin model on a square lattice where each site hosts an
irreducible representation of SU($N$) described by a square Young tableau of
$N/2$ rows and $2S$ columns. We show that negative sign free fermion Monte
Carlo simulations can be carried out for this class of quantum magnets at any
$S$ and even values of $N$. In the large-$N$ limit, the saddle point
approximation favors a four-fold degenerate valence bond solid phase. In the
large $S$-limit, the semi-classical approximation points to N\'eel state. On a
line set by $N=8S + 2$ in the $S$ versus $N$ phase diagram, we observe a
variety of phases proximate to the N\'eel state. At $S = 1/2$ and $3/2$ we
observe the aforementioned four fold degenerate valence bond solid state. At
$S=1$ a two fold degenerate spin nematic state in which the C$_4$ lattice
symmetry is broken down to C$_2$ emerges. Finally at $S=2$ we observe a unique
ground state that pertains to a two-dimensional version of the
Affleck-Kennedy-Lieb-Tasaki state. For our specific realization, this symmetry
protected topological state is characterized by an SU(18), $S=1/2$ boundary
state, that has a dimerized ground state. These phases that are proximate to
the N\'eel state are consistent with the notion of monopole condensation of the
antiferromagnetic order parameter. In particular one expects spin disordered
states with degeneracy set by $\text{mod}(4,2S)$.
