  Stochastic control-flow models (SCFMs) are a class of generative models that
involve branching on choices from discrete random variables. Amortized
gradient-based learning of SCFMs is challenging as most approaches targeting
discrete variables rely on their continuous relaxations---which can be
intractable in SCFMs, as branching on relaxations requires evaluating all
(exponentially many) branching paths. Tractable alternatives mainly combine
REINFORCE with complex control-variate schemes to improve the variance of naive
estimators. Here, we revisit the reweighted wake-sleep (RWS) (Bornschein and
Bengio, 2015) algorithm, and through extensive evaluations, show that it
outperforms current state-of-the-art methods in learning SCFMs. Further, in
contrast to the importance weighted autoencoder, we observe that RWS learns
better models and inference networks with increasing numbers of particles. Our
results suggest that RWS is a competitive, often preferable, alternative for
learning SCFMs.
