  We study the E-string theory on $\mathbb{R}^4\times T^2$ with Wilson lines.
We consider two examples where interesting automorphisms arise. In the first
example, the spectrum is invariant under the $F_4$ Weyl group acting on the
Wilson line parameters. We obtain the Seiberg-Witten curve expressed in terms
of Weyl invariant $F_4$ Jacobi forms. We also clarify how it is related to the
thermodynamic limit of the Nekrasov-type formula. In the second example, the
spectrum is invariant under the $D_4$ triality combined with modular
transformations, the automorphism originally found in the 4d $\mathcal{N}=2$
supersymmetric $\mathrm{SU}(2)$ gauge theory with four massive flavors. We
introduce the notion of triality invariant Jacobi forms and present the
Seiberg-Witten curve in terms of them. We show that this Seiberg-Witten curve
reduces precisely to that of the 4d theory with four flavors in the limit of
$T^2$ shrinking to zero size.
