  The reparameterization trick enables optimizing large scale stochastic
computation graphs via gradient descent. The essence of the trick is to
refactor each stochastic node into a differentiable function of its parameters
and a random variable with fixed distribution. After refactoring, the gradients
of the loss propagated by the chain rule through the graph are low variance
unbiased estimators of the gradients of the expected loss. While many
continuous random variables have such reparameterizations, discrete random
variables lack useful reparameterizations due to the discontinuous nature of
discrete states. In this work we introduce Concrete random
variables---continuous relaxations of discrete random variables. The Concrete
distribution is a new family of distributions with closed form densities and a
simple reparameterization. Whenever a discrete stochastic node of a computation
graph can be refactored into a one-hot bit representation that is treated
continuously, Concrete stochastic nodes can be used with automatic
differentiation to produce low-variance biased gradients of objectives
(including objectives that depend on the log-probability of latent stochastic
nodes) on the corresponding discrete graph. We demonstrate the effectiveness of
Concrete relaxations on density estimation and structured prediction tasks
using neural networks.
