  $f$-divergences, which quantify discrepancy between probability
distributions, are ubiquitous in information theory, machine learning, and
statistics. While there are numerous methods for estimating $f$-divergences
from data, a limit distribution theory, which quantifies fluctuations of the
estimation error, is largely obscure. As limit theorems are pivotal for valid
statistical inference, to close this gap, we develop a general methodology for
deriving distributional limits for $f$-divergences based on the functional
delta method and Hadamard directional differentiability. Focusing on four
prominent $f$-divergences -- Kullback-Leibler divergence, $\chi^2$ divergence,
squared Hellinger distance, and total variation distance -- we identify
sufficient conditions on the population distributions for the existence of
distributional limits and characterize the limiting variables. These results
are used to derive one- and two-sample limit theorems for Gaussian-smoothed
$f$-divergences, both under the null and the alternative. Finally, an
application of the limit distribution theory to auditing differential privacy
is proposed and analyzed for significance level and power against local
alternatives.
