  For each prime $p$, we show that there exist geometrically simple abelian
varieties $A/\mathbb Q$ with non-trivial $p$-torsion in their Tate-Shafarevich
groups. Specifically, for any prime $N\equiv 1 \pmod{p}$, let $A_f$ be an
optimal quotient of $J_0(N)$ with a rational point $P$ of order $p$, and let $B
= A_f/\langle P \rangle$. Then the number of positive integers $d \leq X$, such
that the Tate-Shafarevich group of $\hat B_d$ has non-trivial $p$-torsion, is
$\gg X/\log X$, where $\hat B_d$ is the dual of the $d$-th quadratic twist of
$B$. We prove this more generally for abelian varieties of $\mathrm{GL}_2$-type
with a $p$-isogeny satisfying a mild technical condition. In the special case
of elliptic curves, we give stronger results, including many examples where
$\mathrm{Sha}(E_d)[p] \neq 0$ for an explicit positive proportion of integers
$d$.
