  We show that the inability of a quantum Otto cycle to reach a limit cycle is
connected with the propagator of the cycle being non-compact. For a working
fluid consisting of quantum harmonic oscillators, the transition point in
parameter space where this instability occurs is associated with a
non-hermitian degeneracy (exceptional point) of the eigenvalues of the
propagator. In particular, a third-order exceptional point is observed at the
transition from the region where the eigenvalues are complex numbers to the
region where all the eigenvalues are real. Within this region we find another
exceptional point, this time of second order, at which the trajectory becomes
divergent. The onset of the divergent behavior corresponds to the modulus of
one of the eigenvalues becoming larger than one. The physical origin of this
phenomenon is that the hot and cold heat baths are unable to dissipate the
frictional internal heat generated in the adiabatic strokes of the cycle. This
behavior is contrasted with that of quantum spins as working fluid which have a
compact Hamiltonian and thus no exceptional points. All arguments are
rigorously proved in terms of the systems' associated Lie algebras.
