  We propose Edward, a Turing-complete probabilistic programming language.
Edward defines two compositional representations---random variables and
inference. By treating inference as a first class citizen, on a par with
modeling, we show that probabilistic programming can be as flexible and
computationally efficient as traditional deep learning. For flexibility, Edward
makes it easy to fit the same model using a variety of composable inference
methods, ranging from point estimation to variational inference to MCMC. In
addition, Edward can reuse the modeling representation as part of inference,
facilitating the design of rich variational models and generative adversarial
networks. For efficiency, Edward is integrated into TensorFlow, providing
significant speedups over existing probabilistic systems. For example, we show
on a benchmark logistic regression task that Edward is at least 35x faster than
Stan and 6x faster than PyMC3. Further, Edward incurs no runtime overhead: it
is as fast as handwritten TensorFlow.
