  Estimating the gradients for binary variables is a task that arises
frequently in various domains, such as training discrete latent variable
models. What has been commonly used is a REINFORCE based Monte Carlo estimation
method that uses either independent samples or pairs of negatively correlated
samples. To better utilize more than two samples, we propose ARMS, an
Antithetic REINFORCE-based Multi-Sample gradient estimator. ARMS uses a copula
to generate any number of mutually antithetic samples. It is unbiased, has low
variance, and generalizes both DisARM, which we show to be ARMS with two
samples, and the leave-one-out REINFORCE (LOORF) estimator, which is ARMS with
uncorrelated samples. We evaluate ARMS on several datasets for training
generative models, and our experimental results show that it outperforms
competing methods. We also develop a version of ARMS for optimizing the
multi-sample variational bound, and show that it outperforms both VIMCO and
DisARM. The code is publicly available.
