  Marginal stability is the notion that stability is achieved, but only barely
so. This property constrains the ensemble of configurations explored at low
temperature in a variety of systems, including spin, electron and structural
glasses. A key feature of marginal states is a (saturated) pseudo-gap in the
distribution of soft excitations. We study how such a pseudo-gap appears
dynamically in the case of the Sherrington-Kirkpatrick (SK) spin glass. After
revisiting and correcting the multi-spin-flip criterion for local stability, we
show that stationarity along the hysteresis loop requires that soft spins are
frustrated among each other, with a correlation that diverges as
$C(\lambda)\sim 1/\lambda$, where $\lambda$ is the larger of two considered
local fields. We explain how this arises spontaneously in a marginal system and
develop an analogy between the spin dynamics in the SK model and random walks
in two dimensions. We discuss the applicability of these findings to hard
sphere packings.
