  Objects in a scene are not always related. The execution efficiency of the
one-stage scene graph generation approaches are quite high, which infer the
effective relation between entity pairs using sparse proposal sets and a few
queries. However, they only focus on the relation between subject and object in
triplet set subject entity, predicate entity, object entity, ignoring the
relation between subject and predicate or predicate and object, and the model
lacks self-reasoning ability. In addition, linguistic modality has been
neglected in the one-stage method. It is necessary to mine linguistic modality
knowledge to improve model reasoning ability. To address the above-mentioned
shortcomings, a Self-reasoning Transformer with Visual-linguistic Knowledge
(SrTR) is proposed to add flexible self-reasoning ability to the model. An
encoder-decoder architecture is adopted in SrTR, and a self-reasoning decoder
is developed to complete three inferences of the triplet set, s+o-p, s+p-o and
p+o-s. Inspired by the large-scale pre-training image-text foundation models,
visual-linguistic prior knowledge is introduced and a visual-linguistic
alignment strategy is designed to project visual representations into semantic
spaces with prior knowledge to aid relational reasoning. Experiments on the
Visual Genome dataset demonstrate the superiority and fast inference ability of
the proposed method.
