  We study the modular symmetry in $T^2$ and orbifold comfactifications with
magnetic fluxes. There are $|M|$ zero-modes on $T^2$ with the magnetic flux
$M$. Their wavefunctions as well as massive modes behave as modular forms of
weight $1/2$ and represent the double covering group of $\Gamma \equiv
SL(2,\mathbb{Z})$, $\widetilde{\Gamma} \equiv \widetilde{SL}(2,\mathbb{Z})$.
Each wavefunction on $T^2$ with the magnetic flux $M$ transforms under
$\widetilde{\Gamma}(2|M|)$, which is the normal subgroup of
$\widetilde{SL}(2,\mathbb{Z})$. Then, $|M|$ zero-modes are representations of
the quotient group $\widetilde{\Gamma}'_{2|M|} \equiv
\widetilde{\Gamma}/\widetilde{\Gamma}(2|M|)$. We also study the modular
symmetry on twisted and shifted orbifolds $T^2/\mathbb{Z}_N$. Wavefunctions are
decomposed into smaller representations by eigenvalues of twist and shift. They
provide us with reduction of reducible representations on $T^2$.
