  It is known that any Galois representation $\rho : G_{\mathbb{Q}} \rightarrow
\mathrm{GL}(2,\mathbb{F}_p)$ with determinant equal to the mod-$p$ cyclotomic
character, arises from the $p$-torsion of an elliptic curve over $\mathbb{Q}$,
if and only if $p \leq 5$. In dimension $g = 2$, when $p \le 3$, it is again
known that any Galois representation valued in $\mathrm{GSp}(4,\mathbb{F}_p)$
with cyclotomic similitude character arises from an abelian surface. In this
paper, we study this question for all primes $p$ and dimensions $g \ge 2$. When
$g \ge 2$ and $(g,p) \neq (2,2)$, $(2,3)$, $(3,2)$, we prove the existence of a
Galois representation over $\mathbb{Q}$ valued in
$\mathrm{GSp}(2g,\mathbb{F}_p)$ with cyclotomic similitude character, that
cannot arise as the $p$-torsion representation of any $g$-dimensional abelian
variety over $\mathbb{Q}$.
