  This paper proposes an efficient method for computing selected generalized
eigenpairs of a sparse Hermitian definite matrix pencil $(A,B)$. Based on
Zolotarev's best rational function approximations of the signum function and
conformal mapping techniques, we construct the best rational function
approximation of a rectangular function supported on an arbitrary interval via
function compositions with partial fraction representations. This new best
rational function approximation can be applied to construct spectrum filters of
$(A,B)$ with a smaller number of poles than a direct construction without
function compositions. Combining fast direct solvers and the shift-invariant
generalized minimal residual method, a hybrid fast algorithm is proposed to
apply spectral filters efficiently. Compared to the state-of-the-art algorithm
FEAST, the proposed rational function approximation is more efficient when
sparse matrix factorizations are required to solve multi-shift linear systems
in the eigensolver, since the smaller number of matrix factorizations is needed
in our method. The efficiency and stability of the proposed method are
demonstrated by numerical examples from computational chemistry.
