  Generative models (GMs) such as Generative Adversary Network (GAN) and
Variational Auto-Encoder (VAE) have thrived these years and achieved high
quality results in generating new samples. Especially in Computer Vision, GMs
have been used in image inpainting, denoising and completion, which can be
treated as the inference from observed pixels to corrupted pixels. However,
images are hierarchically structured which are quite different from many
real-world inference scenarios with non-hierarchical features. These inference
scenarios contain heterogeneous stochastic variables and irregular mutual
dependences. Traditionally they are modeled by Bayesian Network (BN). However,
the learning and inference of BN model are NP-hard thus the number of
stochastic variables in BN is highly constrained. In this paper, we adapt
typical GMs to enable heterogeneous learning and inference in polynomial
time.We also propose an extended autoregressive (EAR) model and an EAR with
adversary loss (EARA) model and give theoretical results on their
effectiveness. Experiments on several BN datasets show that our proposed EAR
model achieves the best performance in most cases compared to other GMs. Except
for black box analysis, we've also done a serial of experiments on Markov
border inference of GMs for white box analysis and give theoretical results.
