  Waves in a variety of fields in physics, such as mechanics, optics,
spintronics, and nonlinear systems, obey generalized eigenvalue equations. To
study non-Hermitian physics of those systems, in this paper, we construct a
non-Bloch band theory of generalized eigenvalue problems. Specifically, we show
that eigenvalues of a transfer matrix lead to a certain condition imposed on
the generalized Brillouin zone, which allows us to develop a theory to
calculate the continuum bands. As a concrete example, we examine the
non-Hermitian skin effect of photonic crystals composed of chiral metamaterials
by invoking our theoretical framework. When the medium has circularly polarized
eigenmodes, we find that each eigenmode localizes at either of the edges,
depending on whether it is left- or right-circularly polarized. In contrast,
when the medium only has linearly polarized eigenmodes, every eigenmode
localizes to the edge of the same side independent of its polarization. We
demonstrate that the localization lengths of those eigenmodes can be determined
from the chiral parameters and eigenfrequencies of the photonic crystal.
