  The issue of hallucinations is a prevalent concern in existing Large
Vision-Language Models (LVLMs). Previous efforts have primarily focused on
investigating object hallucinations, which can be easily alleviated by
introducing object detectors. However, these efforts neglect hallucinations in
inter-object relationships, which is essential for visual comprehension. In
this work, we introduce R-Bench, a novel benchmark for evaluating Vision
Relationship Hallucination. R-Bench features image-level questions that focus
on the existence of relationships and instance-level questions that assess
local visual comprehension. We identify three types of relationship
co-occurrences that lead to hallucinations: relationship-relationship,
subject-relationship, and relationship-object. The visual instruction tuning
dataset's long-tail distribution significantly impacts LVLMs' understanding of
visual relationships. Furthermore, our analysis reveals that current LVLMs tend
to disregard visual content and overly rely on the common sense knowledge of
Large Language Models. They also struggle with reasoning about spatial
relationships based on contextual information.
