  We study the relationship of the spectral form factor with quantum as well as
classical probabilities to return. Defining a quantum return probability in
phase space as a trace over the propagator of the Wigner function allows us to
identify and resolve manifolds in phase space that contribute to the form
factor. They can be associated to classical invariant manifolds such as
periodic orbits, but also to non-classical structures like sets of midpoints
between periodic points. By contrast to scars in wavefunctions, these features
are not subject to the uncertainty relation and therefore need not show any
smearing. They constitute important exceptions from a continuous convergence in
the classical limit of the Wigner towards the Liouville propagator. We support
our theory with numerical results for the quantum cat map and the harmonically
driven quartic oscillator.
