  This thesis studies high-dimensional, continuous-valued pairwise Markov
Random Fields. We are particularly interested in approximating pairwise
densities whose logarithm belongs to a Sobolev space. For this problem we
propose the method of exponential series [Crain, 1974; Barron and Sheu, 1991],
which approximates the log density by a finite- dimensional exponential family
with the number of sufficient statistics increasing with the sample size. We
consider two approaches to estimating these models. The first is regularized
maximum likelihood. This involves optimizing the sum of the log-likelihood of
the data and a sparsity-inducing regularizer. We provide consistency and edge
selection guarantees for this method. We then propose a variational
approximation to the likelihood based on tree- reweighted, nonparametric
message passing.
  We then consider estimation using regularized score matching. This approach
uses an alternative scoring rule to the log-likelihood, which obviates the need
to compute the normalizing constant of the distribution. For general
continuous-valued exponential families, we provide parameter and edge
consistency results. We then describe results for model selection in the
nonparametric pairwise model using exponential series. The regularized score
matching problem is shown to be a convex program; we provide scalable
algorithms based on consensus Alternating Direction Method of Multipliers
(ADMM, [Boyd et al., 2011]) and Coordinate-wise Descent. We compare our method
to others in the literature as well as the aforementioned TRW estimator using
simulated data.
