  A wide variety of generative models for graphs have been proposed. They are
used in drug discovery, road networks, neural architecture search, and program
synthesis. Generating graphs has theoretical challenges, such as isomorphic
representations -- evaluating how well a generative model performs is
difficult. Which model to choose depending on the application domain?
  We extensively study kernel-based metrics on distributions of graph
invariants and manifold-based and kernel-based metrics in graph embedding
space. Manifold-based metrics outperform kernel-based metrics in embedding
space. We use these metrics to compare GraphRNN and GRAN, two well-known
generative models for graphs, and unveil the influence of node orderings. It
shows the superiority of GRAN over GraphRNN - further, our proposed adaptation
of GraphRNN with a depth-first search ordering is effective for small-sized
graphs.
  A guideline on good practices regarding dataset selection and node feature
initialization is provided. Our work is accompanied by open-source code and
reproducible experiments.
