  We present a novel graph Transformer generative adversarial network (GTGAN)
to learn effective graph node relations in an end-to-end fashion for the
challenging graph-constrained house generation task. The proposed
graph-Transformer-based generator includes a novel graph Transformer encoder
that combines graph convolutions and self-attentions in a Transformer to model
both local and global interactions across connected and non-connected graph
nodes. Specifically, the proposed connected node attention (CNA) and
non-connected node attention (NNA) aim to capture the global relations across
connected nodes and non-connected nodes in the input graph, respectively. The
proposed graph modeling block (GMB) aims to exploit local vertex interactions
based on a house layout topology. Moreover, we propose a new node
classification-based discriminator to preserve the high-level semantic and
discriminative node features for different house components. Finally, we
propose a novel graph-based cycle-consistency loss that aims at maintaining the
relative spatial relationships between ground truth and predicted graphs.
Experiments on two challenging graph-constrained house generation tasks (i.e.,
house layout and roof generation) with two public datasets demonstrate the
effectiveness of GTGAN in terms of objective quantitative scores and subjective
visual realism. New state-of-the-art results are established by large margins
on both tasks.
