  We show that only finitely many complex genus two curves and four punctured
spheres admit rank two local systems of geometric origin, and moreover each
carries finitely many. This gives further counterexamples to a conjecture of
Esnault and Kerz: counterexamples over very general curves were recently
obtained by Landesman and Litt. In the second part we prove an analogue of this
result in positive characteristic, namely that over $\overline{\mathbb{F}}_p$,
only finitely many genus two curves admit non-trivial rank two local systems
pulled back from a fixed quaternionic Shimura variety, and the same for
$\mathbb{P}^1$ minus four points; conjecturally, every rank two local system
arises as such a pullback. This provides results towards Maeda's conjecture on
Galois orbits of eigenforms over function fields. The proofs make use of ideas
from the work of Landesman and Litt such as isomonodromy, as well as crucially
the description of the Goren-Oort strata due to Tian and Xiao.
