  We study the partial breaking of $N=2$ rigid supersymmetry for a generic
rigid special geometry of $n$ abelian vector multiplets in the presence of
Fayet-Iliopoulos terms induced by the Hyper-K\"ahler momentum map. By
exhibiting the symplectic structure of the problem we give invariant conditions
for the breaking to occur, which rely on a quartic invariant of the
Fayet-Iliopoulos charges as well as on a modification of the $N=2$ rigid
symmetry algebra by a vector central charge.
