  We derive a new variational formula for the R\'enyi family of divergences,
$R_\alpha(Q\|P)$, between probability measures $Q$ and $P$. Our result
generalizes the classical Donsker-Varadhan variational formula for the
Kullback-Leibler divergence. We further show that this R\'enyi variational
formula holds over a range of function spaces; this leads to a formula for the
optimizer under very weak assumptions and is also key in our development of a
consistency theory for R\'enyi divergence estimators. By applying this theory
to neural-network estimators, we show that if a neural network family satisfies
one of several strengthened versions of the universal approximation property
then the corresponding R\'enyi divergence estimator is consistent. In contrast
to density-estimator based methods, our estimators involve only expectations
under $Q$ and $P$ and hence are more effective in high dimensional systems. We
illustrate this via several numerical examples of neural network estimation in
systems of up to 5000 dimensions.
