  The late-time behavior of spectral form factor (SFF) encodes the inherent
discreteness of a quantum system, which should be generically non-vanishing. We
study an index analog of the microcanonical spectrum form factor in
four-dimensional $\mathcal{N}=4$ super Yang-Mills theory. In the large $N$
limit and at large enough energy, the most dominant saddle corresponds to the
black hole in the AdS bulk. This gives rise to the slope that decreases
exponentially for a small imaginary chemical potential, which is a natural
analog of an early time. We find that the `late-time' behavior is governed by
the multi-cut saddles that arise in the index matrix model, which are
non-perturbatively sub-dominant at early times. These saddles become dominant
at late times, preventing the SFF from decaying. These multi-cut saddles
correspond to the orbifolded Euclidean black holes in the AdS bulk, therefore
giving the geometrical interpretation of the `ramp.' Our analysis is done in
the standard AdS/CFT setting without ensemble average or wormholes.
