  A common measure of a function's complexity is the count of its stationary
points. For complicated functions, this count grows exponentially with the
volume and dimension of their domain. In practice, the count is averaged over a
class of functions (the annealed average), but the large numbers involved can
produce averages biased by extremely rare samples. Typical counts are reliably
found by taking the average of the logarithm (the quenched average), which is
more difficult and not often done in practice. When most stationary points are
uncorrelated with each other, quenched and anneals averages are equal.
Equilibrium heuristics can guarantee when most of the lowest minima will be
uncorrelated. We show that these equilibrium heuristics cannot be used to draw
conclusions about other minima and saddles by producing examples among
Gaussian-correlated functions on the hypersphere where the count of certain
saddles and minima has different quenched and annealed averages, despite being
guaranteed `safe' in the equilibrium setting. We determine conditions for the
emergence of nontrivial correlations between saddles, and discuss the
implications for the geometry of those functions and what out-of-equilibrium
settings might be affected.
