  The fidelity susceptibility is a tool for studying quantum phase transitions
in the Hermitian condensed matter systems. Recently, it has been generalized
with the biorthogonal basis for the non-Hermitian quantum systems. From the
general perturbation description with the constraint of parity-time (PT)
symmetry, we show that the fidelity $\mathcal{F}$ is always real for the
PT-unbroken states. For the PT-broken states, the real part of the fidelity
susceptibility $\mathrm{Re}[\mathcal{X}_F]$ is corresponding to considering
both the PT partner states, and the negative infinity is explored by the
perturbation theory when the parameter approaches the exceptional point (EP).
Moreover, at the second-order EP, we prove that the real part of the fidelity
between PT-unbroken and PT-broken states is
$\mathrm{Re}\mathcal{F}=\frac{1}{2}$. Based on these general properties, we
study the two-legged non-Hermitian Su-Schrieffer-Heeger (SSH) model and the
non-Hermitian XXZ spin chain. We find that for both interacting and
non-interacting systems, the real part of fidelity susceptibility density goes
to negative infinity when the parameter approaches the EP, and verifies it is a
second-order EP by $\mathrm{Re}\mathcal{F}=\frac{1}{2}$.
