  We prove that the direct image complex for the $D$-twisted $SL_n$ Hitchin
fibration is determined by its restriction to the elliptic locus, where the
spectral curves are integral. The analogous result for $GL_n$ is due to P.-H.
Chaudouard and G. Laumon. Along the way, we prove that the Tate module of the
relative Prym group scheme is polarizable, and we also prove
$\delta$-regularity results for some auxiliary weak abelian fibrations.
