  We define the notion of a $G$-structure for elliptic curves, where $G$ is a
finite 2-generated group. When $G$ is abelian, a $G$-structure is the same as a
classical congruence level structure. There is a natural action of
$\text{SL}_2(\mathbb{Z})$ on these level structures. If $\Gamma$ is a
stabilizer of this action, then the quotient of the upper half plane by
$\Gamma$ parametrizes isomorphism classes of elliptic curves equipped with
$G$-structures. When $G$ is "sufficiently" nonabelian, the stabilizers $\Gamma$
are noncongruence. As a result we realize noncongruence modular curves as
moduli spaces of elliptic curves equipped with nonabelian $G$-structures. As
applications we describe links to the Inverse Galois Problem, and show how our
moduli interpretations explains the bad primes for the Unbounded Denominators
Conjecture, and allows us to translate the conjecture into the language of
geometry and Galois theory.
