  Although the expansion of the Universe explicitly breaks the time-translation
symmetry, cosmological predictions for the stochastic gravitational wave
background (SGWB) are usually derived under the so-called stationary
hypothesis. By dropping this assumption and keeping track of the time
dependence of gravitational waves at all length scales, we derive the expected
unequal-time (and equal-time) waveform of the SGWB generated by scaling
sources, such as cosmic defects. For extinct and smooth enough sources, we show
that all observable quantities are uniquely and analytically determined by the
holomorphic Fourier transform of the anisotropic stress correlator. Both the
strain power spectrum and the energy density parameter are shown to have an
oscillatory fine structure, they significantly differ on large scales while
running in phase opposition at large wavenumbers $k$. We then discuss scaling
sources that are never extinct nor smooth and which generate a singular Fourier
transform of the anisotropic stress correlator. For these, we find the
appearance of interferences on top of the above-mentioned fine-structure as
well as atypical behaviour at small scales. For instance, we expect the
rescaled strain power spectrum $k^2 \mathcal{P}_h$ generated by long cosmic
strings in the matter era to oscillate around a scale invariant plateau. These
singular sources are also shown to produce orders of magnitude difference
between the rescaled strain spectra and the energy density parameter suggesting
that only the former should be used for making reliable observable predictions.
Finally, we discuss how measuring such a fine structure in the SGWB could
disambiguate the possible cosmological sources.
