  Undirected graphical models are applied in genomics, protein structure
prediction, and neuroscience to identify sparse interactions that underlie
discrete data. Although Bayesian methods for inference would be favorable in
these contexts, they are rarely used because they require doubly intractable
Monte Carlo sampling. Here, we develop a framework for scalable Bayesian
inference of discrete undirected models based on two new methods. The first is
Persistent VI, an algorithm for variational inference of discrete undirected
models that avoids doubly intractable MCMC and approximations of the partition
function. The second is Fadeout, a reparameterization approach for variational
inference under sparsity-inducing priors that captures a posteriori
correlations between parameters and hyperparameters with noncentered
parameterizations. We find that, together, these methods for variational
inference substantially improve learning of sparse undirected graphical models
in simulated and real problems from physics and biology.
