  In panoptic scene graph generation (PSGG), models retrieve interactions
between objects in an image which are grounded by panoptic segmentation masks.
Previous evaluations on panoptic scene graphs have been subject to an erroneous
evaluation protocol where multiple masks for the same object can lead to
multiple relation distributions per mask-mask pair. This can be exploited to
increase the final score. We correct this flaw and provide a fair ranking over
a wide range of existing PSGG models. The observed scores for existing methods
increase by up to 7.4 mR@50 for all two-stage methods, while dropping by up to
19.3 mR@50 for all one-stage methods, highlighting the importance of a correct
evaluation. Contrary to recent publications, we show that existing two-stage
methods are competitive to one-stage methods. Building on this, we introduce
the Decoupled SceneFormer (DSFormer), a novel two-stage model that outperforms
all existing scene graph models by a large margin of +11 mR@50 and +10 mNgR@50
on the corrected evaluation, thus setting a new SOTA. As a core design
principle, DSFormer encodes subject and object masks directly into feature
space.
