  We prove that the Fibonacci quantum representations
$\rho_{g,n}:\rm{Mod}_{g,n}\to \rm{PU}(p,q)$ for
$(g,n)\in\{(0,4),(0,5),(1,2),(1,3),(2,1)\}$ are holonomy representations of
complex hyperbolic structures on some compactifications of the corresponding
moduli spaces $\mathcal{M}_{g,n}$. As a corollary, the forgetful map between
the corresponding compactifications of $\mathcal M_{1,3}$ and $\mathcal
M_{1,2}$ is a surjective holomorphic map between compact complex hyperbolic
orbifolds of different dimensions higher than one, giving an answer to a
problem raised by Siu.
  The proof consists in computing their Toledo invariants: we put this
computation in a broader context, replacing the Fibonacci representations with
any Hermitian modular functor and extending the Toledo invariant to a full
series of cohomological invariants beginning with the signature $p-q$.
  We prove that these invariants satisfy the axioms of a Cohomological Field
Theory and compute the $R$-matrix at first order (hence the usual Toledo
invariants) in the case of the $\rm{SU}_2/\rm{SO}_3$-quantum representations at
any level.
