  Suppose $F$ is either a global field or a finitely generated extension of
${\mathbf Q}$, $A$ is an abelian variety over $F$, and $\ell$ is a prime not
equal to the characteristic of $F$. Let $Z$ denote the center of the
endomorphism algebra of $A$. Let $G$ denote the group of ${\mathbf
Q}_\ell$-points of the identity connected component of the Zariski closure of
the image of the $\ell$-adic representation associated to $A$. We prove the
$\ell$-independence of the intersection of $G$ with the torsion subgroup of
$Z$. Our results provide evidence in the direction of the Mumford-Tate
Conjecture.
