  In this paper we investigate the problems of class imbalance and irrelevant
relationships in Visual Relationship Detection (VRD). State-of-the-art deep VRD
models still struggle to predict uncommon classes, limiting their
applicability. Moreover, many methods are incapable of properly filtering out
background relationships while predicting relevant ones. Although these
problems are very apparent, they have both been overlooked so far. We analyse
why this is the case and propose modifications to both model and training to
alleviate the aforementioned issues, as well as suggesting new measures to
complement existing ones and give a more holistic picture of the efficacy of a
model.
