  To backpropagate the gradients through stochastic binary layers, we propose
the augment-REINFORCE-merge (ARM) estimator that is unbiased, exhibits low
variance, and has low computational complexity. Exploiting variable
augmentation, REINFORCE, and reparameterization, the ARM estimator achieves
adaptive variance reduction for Monte Carlo integration by merging two
expectations via common random numbers. The variance-reduction mechanism of the
ARM estimator can also be attributed to either antithetic sampling in an
augmented space, or the use of an optimal anti-symmetric "self-control"
baseline function together with the REINFORCE estimator in that augmented
space. Experimental results show the ARM estimator provides state-of-the-art
performance in auto-encoding variational inference and maximum likelihood
estimation, for discrete latent variable models with one or multiple stochastic
binary layers. Python code for reproducible research is publicly available.
