  In experiments where one searches a large parameter space for an anomaly, one
often finds many spurious noise-induced peaks in the likelihood. This is known
as the look-elsewhere effect, and must be corrected for when performing
statistical analysis. This paper introduces a method to calibrate the false
alarm probability (FAP), or $p$-value, for a given dataset by considering the
heights of the highest peaks in the likelihood. In the simplest form of
self-calibration, the look-elsewhere-corrected $\chi^2$ of a physical peak is
approximated by the $\chi^2$ of the peak minus the $\chi^2$ of the highest
noise-induced peak. Generalizing this concept to consider lower peaks provides
a fast method to quantify the statistical significance with improved accuracy.
In contrast to alternative methods, this approach has negligible computational
cost as peaks in the likelihood are a byproduct of every peak-search analysis.
We apply to examples from astronomy, including planet detection, periodograms,
and cosmology.
