  Several techniques have recently aimed to improve the performance of deep
learning models for Scene Graph Generation (SGG) by incorporating background
knowledge. State-of-the-art techniques can be divided into two families: one
where the background knowledge is incorporated into the model in a subsymbolic
fashion, and another in which the background knowledge is maintained in
symbolic form. Despite promising results, both families of techniques face
several shortcomings: the first one requires ad-hoc, more complex neural
architectures increasing the training or inference cost; the second one suffers
from limited scalability w.r.t. the size of the background knowledge. Our work
introduces a regularization technique for injecting symbolic background
knowledge into neural SGG models that overcomes the limitations of prior art.
Our technique is model-agnostic, does not incur any cost at inference time, and
scales to previously unmanageable background knowledge sizes. We demonstrate
that our technique can improve the accuracy of state-of-the-art SGG models, by
up to 33%.
