  A generalization of the Apery-like numbers, which is used to describe the
special values $\zeta_Q(2)$ and $\zeta_Q(3)$ of the spectral zeta function for
the non-commutative harmonic oscillator, are introduced and studied. In fact,
we give a recurrence relation for them, which shows a ladder structure among
them. Further, we consider the `rational part' of the higher Apery-like
numbers. We discuss several kinds of congruence relations among them, which are
regarded as an analogue of the ones among Apery numbers.
