  Estimating the ratio of two probability densities from finitely many
observations of the densities is a central problem in machine learning and
statistics with applications in two-sample testing, divergence estimation,
generative modeling, covariate shift adaptation, conditional density
estimation, and novelty detection. In this work, we analyze a large class of
density ratio estimation methods that minimize a regularized Bregman divergence
between the true density ratio and a model in a reproducing kernel Hilbert
space (RKHS). We derive new finite-sample error bounds, and we propose a
Lepskii type parameter choice principle that minimizes the bounds without
knowledge of the regularity of the density ratio. In the special case of
quadratic loss, our method adaptively achieves a minimax optimal error rate. A
numerical illustration is provided.
