  Motivated by the recent growing interest in the field of
$\mathcal{P}\mathcal{T}$-symmetric Hamiltonian systems we theoretically study
the emergency of singularities called Exceptional Points ($\textit{EP}$s) in
the eigenspectrum of pseudo-Hermitian Hamiltonian as the strength of
Hermiticity-breaking terms turns on. Using general symmetry arguments, we
characterize the separate energy levels by a topological $\mathbb{Z}_2$ index
which corresponds to the signs $\pm 1$ of the eigenvalues of pseudo-metric
operator $\hat \zeta$ in the absence of Hermiticity-breaking terms. After that,
we show explicitly that the formation of second-order $\textit{EP}$s is
governed by this $\mathbb{Z}_2$-index: only the pairs of levels with
$\textit{opposite}$ index can provide second-order $\textit{EP}$s. Our general
analysis is accompanied by a detailed study of $\textit{EP}$s appearance in an
exemplary $\mathcal{P}\mathcal{T}$-symmetric pseudo-Hermitian system with
parity operator in the role of $\hat \zeta$: a transverse-field Ising spin
chain with a staggered imaginary longitudinal field. Using analytically
computed parity indices of all the levels, we analyze the eigenspectrum of the
model in general, and the formation of third-order $\textit{EP}$s in particular
