  Efficient modeling of relational data arising in physical, social, and
information sciences is challenging due to complicated dependencies within the
data. In this work, we build off of semi-implicit graph variational
auto-encoders to capture higher-order statistics in a low-dimensional graph
latent representation. We incorporate hyperbolic geometry in the latent space
through a Poincare embedding to efficiently represent graphs exhibiting
hierarchical structure. To address the naive posterior latent distribution
assumptions in classical variational inference, we use semi-implicit
hierarchical variational Bayes to implicitly capture posteriors of given graph
data, which may exhibit heavy tails, multiple modes, skewness, and highly
correlated latent structures. We show that the existing semi-implicit
variational inference objective provably reduces information in the observed
graph. Based on this observation, we estimate and add an additional mutual
information term to the semi-implicit variational inference learning objective
to capture rich correlations arising between the input and latent spaces. We
show that the inclusion of this regularization term in conjunction with the
Poincare embedding boosts the quality of learned high-level representations and
enables more flexible and faithful graphical modeling. We experimentally
demonstrate that our approach outperforms existing graph variational
auto-encoders both in Euclidean and in hyperbolic spaces for edge link
prediction and node classification.
