  Deep neural networks are powerful parametric models that can be trained
efficiently using the backpropagation algorithm. Stochastic neural networks
combine the power of large parametric functions with that of graphical models,
which makes it possible to learn very complex distributions. However, as
backpropagation is not directly applicable to stochastic networks that include
discrete sampling operations within their computational graph, training such
networks remains difficult. We present MuProp, an unbiased gradient estimator
for stochastic networks, designed to make this task easier. MuProp improves on
the likelihood-ratio estimator by reducing its variance using a control variate
based on the first-order Taylor expansion of a mean-field network. Crucially,
unlike prior attempts at using backpropagation for training stochastic
networks, the resulting estimator is unbiased and well behaved. Our experiments
on structured output prediction and discrete latent variable modeling
demonstrate that MuProp yields consistently good performance across a range of
difficult tasks.
