  We develop the formalism of the finite modular group $\Gamma'_4 \equiv S'_4$,
a double cover of the modular permutation group $\Gamma_4 \simeq S_4$, for
theories of flavour. The integer weight $k>0$ of the level 4 modular forms
indispensable for the formalism can be even or odd. We explicitly construct the
lowest-weight ($k=1$) modular forms in terms of two Jacobi theta constants,
denoted as $\varepsilon(\tau)$ and $\theta(\tau)$, $\tau$ being the modulus. We
show that these forms furnish a 3D representation of $S'_4$ not present for
$S_4$. Having derived the $S'_4$ multiplication rules and Clebsch-Gordan
coefficients, we construct multiplets of modular forms of weights up to $k=10$.
These are expressed as polynomials in $\varepsilon$ and $\theta$, bypassing the
need to search for non-linear constraints. We further show that within $S'_4$
there are two options to define the (generalised) CP transformation and we
discuss the possible residual symmetries in theories based on modular and CP
invariance. Finally, we provide two examples of application of our results,
constructing phenomenologically viable lepton flavour models.
