  Why do many modern neural-network-based graph generative models fail to
reproduce typical real-world network characteristics, such as high triangle
density? In this work we study the limitations of edge independent random graph
models, in which each edge is added to the graph independently with some
probability. Such models include both the classic Erd\"{o}s-R\'{e}nyi and
stochastic block models, as well as modern generative models such as NetGAN,
variational graph autoencoders, and CELL. We prove that subject to a bounded
overlap condition, which ensures that the model does not simply memorize a
single graph, edge independent models are inherently limited in their ability
to generate graphs with high triangle and other subgraph densities. Notably,
such high densities are known to appear in real-world social networks and other
graphs. We complement our negative results with a simple generative model that
balances overlap and accuracy, performing comparably to more complex models in
reconstructing many graph statistics.
