  Rejection sampling methods have recently been proposed to improve the
performance of discriminator-based generative models. However, these methods
are only optimal under an unlimited sampling budget, and are usually applied to
a generator trained independently of the rejection procedure. We first propose
an Optimal Budgeted Rejection Sampling (OBRS) scheme that is provably optimal
with respect to \textit{any} $f$-divergence between the true distribution and
the post-rejection distribution, for a given sampling budget. Second, we
propose an end-to-end method that incorporates the sampling scheme into the
training procedure to further enhance the model's overall performance. Through
experiments and supporting theory, we show that the proposed methods are
effective in significantly improving the quality and diversity of the samples.
