  Scene graph generation (SGG) aims to detect objects and predict the
relationships between each pair of objects. Existing SGG methods usually suffer
from several issues, including 1) ambiguous object representations, as graph
neural network-based message passing (GMP) modules are typically sensitive to
spurious inter-node correlations, and 2) low diversity in relationship
predictions due to severe class imbalance and a large number of missing
annotations. To address both problems, in this paper, we propose a regularized
unrolling network (RU-Net). We first study the relation between GMP and graph
Laplacian denoising (GLD) from the perspective of the unrolling technique,
determining that GMP can be formulated as a solver for GLD. Based on this
observation, we propose an unrolled message passing module and introduce an
$\ell_p$-based graph regularization to suppress spurious connections between
nodes. Second, we propose a group diversity enhancement module that promotes
the prediction diversity of relationships via rank maximization. Systematic
experiments demonstrate that RU-Net is effective under a variety of settings
and metrics. Furthermore, RU-Net achieves new state-of-the-arts on three
popular databases: VG, VRD, and OI. Code is available at
https://github.com/siml3/RU-Net.
