  We develop unbiased implicit variational inference (UIVI), a method that
expands the applicability of variational inference by defining an expressive
variational family. UIVI considers an implicit variational distribution
obtained in a hierarchical manner using a simple reparameterizable distribution
whose variational parameters are defined by arbitrarily flexible deep neural
networks. Unlike previous works, UIVI directly optimizes the evidence lower
bound (ELBO) rather than an approximation to the ELBO. We demonstrate UIVI on
several models, including Bayesian multinomial logistic regression and
variational autoencoders, and show that UIVI achieves both tighter ELBO and
better predictive performance than existing approaches at a similar
computational cost.
