  The fidelity susceptibility has been used to detect quantum phase transitions
in the Hermitian quantum many-body systems over a decade, where the fidelity
susceptibility density approaches $+\infty$ in the thermodynamic limits. Here
the fidelity susceptibility $\chi$ is generalized to non-Hermitian quantum
systems by taking the geometric structure of the Hilbert space into
consideration. Instead of solving the metric equation of motion from scratch,
we chose a gauge where the fidelities are composed of biorthogonal eigenstates
and can be worked out algebraically or numerically when not on the exceptional
point (EP). Due to the properties of the Hilbert space geometry at EP, we found
that EP can be found when $\chi$ approaches $-\infty$. As examples, we
investigate the simplest $\mathcal{PT}$ symmetric $2\times2$ Hamiltonian with a
single tuning parameter and the non-Hermitian Su-Schriffer-Heeger model.
