  We derive the potential modular symmetries of heterotic string theory. For a
toroidal compactification with Wilson line modulus, we obtain the Siegel
modular group $\mathrm{Sp}(4,\mathbb{Z})$ that includes the modular symmetries
$\mathrm{SL}(2,\mathbb{Z})_T$ and $\mathrm{SL}(2,\mathbb{Z})_U$ (of the
"geometric" moduli $T$ and $U$) as well as mirror symmetry. In addition, string
theory provides a candidate for a CP-like symmetry that enhances the Siegel
modular group to $\mathrm{GSp}(4,\mathbb{Z})$.
