  Quantum fluctuations of the N\'eel state of the square lattice
antiferromagnet are usually described by a $\mathbb{CP}^1$ theory of bosonic
spinons coupled to a U(1) gauge field, and with a global SU(2) spin rotation
symmetry. Such a theory also has a confining phase with valence bond solid
(VBS) order, and upon including spin-singlet charge 2 Higgs fields, deconfined
phases with $\mathbb{Z}_2$ topological order possibly intertwined with discrete
broken global symmetries. We present dual theories of the same phases starting
from a mean-field theory of fermionic spinons moving in $\pi$-flux in each
square lattice plaquette. Fluctuations about this $\pi$-flux state are
described by 2+1 dimensional quantum chromodynamics (QCD$_3$) with a SU(2)
gauge group and $N_f=2$ flavors of massless Dirac fermions. It has recently
been argued by Wang et al. (arXiv:1703.02426) that this QCD$_3$ theory
describes the N\'eel-VBS quantum phase transition. We introduce adjoint Higgs
fields in QCD$_3$, and obtain fermionic dual descriptions of the phases with
$\mathbb{Z}_2$ topological order obtained earlier using the bosonic
$\mathbb{CP}^1$ theory. We also present a fermionic spinon derivation of the
monopole Berry phases in the U(1) gauge theory of the VBS state. The global
phase diagram of these phases contains multi-critical points, and our results
imply new boson-fermion dualities between critical gauge theories of these
points.
