  Discrete variational auto-encoders (VAEs) are able to represent semantic
latent spaces in generative learning. In many real-life settings, the discrete
latent space consists of high-dimensional structures, and propagating gradients
through the relevant structures often requires enumerating over an
exponentially large latent space. Recently, various approaches were devised to
propagate approximated gradients without enumerating over the space of possible
structures. In this work, we use Natural Evolution Strategies (NES), a class of
gradient-free black-box optimization algorithms, to learn discrete structured
VAEs. The NES algorithms are computationally appealing as they estimate
gradients with forward pass evaluations only, thus they do not require to
propagate gradients through their discrete structures. We demonstrate
empirically that optimizing discrete structured VAEs using NES is as effective
as gradient-based approximations. Lastly, we prove NES converges for
non-Lipschitz functions as appear in discrete structured VAEs.
