  Probabilistic graphical models that encode an underlying Markov random field
are fundamental building blocks of generative modeling to learn latent
representations in modern multivariate data sets with complex dependency
structures. Among these, the exponential family graphical models are especially
popular, given their fairly well-understood statistical properties and
computational scalability to high-dimensional data based on pseudo-likelihood
methods. These models have been successfully applied in many fields, such as
the Ising model in statistical physics and count graphical models in genomics.
Another strand of models allows some nodes to be latent, so as to allow the
marginal distribution of the observable nodes to depart from exponential family
to capture more complex dependence. These approaches form the basis of
generative models in artificial intelligence, such as the Boltzmann machines
and their restricted versions. A fundamental barrier to likelihood-based (i.e.,
both maximum likelihood and fully Bayesian) inference in both fully and
partially observed cases is the intractability of the likelihood. The usual
workaround is via adopting pseudo-likelihood based approaches, following the
pioneering work of Besag (1974). The goal of this paper is to demonstrate that
full likelihood based analysis of these models is feasible in a computationally
efficient manner. The chief innovation lies in using a technique of Geyer
(1991) to estimate the intractable normalizing constant, as well as its
gradient, for intractable graphical models. Extensive numerical results,
supporting theory and comparisons with pseudo-likelihood based approaches
demonstrate the applicability of the proposed method.
