  Given the remarkable success that large visual language models (LVLMs) have
achieved in image perception tasks, the endeavor to make LVLMs perceive the
world like humans is drawing increasing attention. Current multi-modal
benchmarks primarily focus on facts or specific topic-related knowledge
contained within individual images. However, they often overlook the
associative relations between multiple images, which require the identification
and analysis of similarities among entities or content present in different
images. Therefore, we propose the multi-image relation association task and a
meticulously curated Multi-granularity Multi-image Relational Association
(MMRA) benchmark, comprising 1,024 samples. In order to systematically and
comprehensively evaluate current LVLMs, we establish an associational relation
system among images that contain 11 subtasks (e.g, UsageSimilarity, SubEvent)
at two granularity levels (i.e., image and entity) according to the relations
in ConceptNet. Our experiments reveal that on the MMRA benchmark, current
multi-image LVLMs exhibit distinct advantages and disadvantages across various
subtasks. Notably, fine-grained, entity-level multi-image perception tasks pose
a greater challenge for LVLMs compared to image-level tasks. Moreover, LVLMs
perform poorly on spatial-related tasks, indicating that LVLMs still have
limited spatial awareness. Additionally, our findings indicate that while LVLMs
demonstrate a strong capability to perceive image details, enhancing their
ability to associate information across multiple images hinges on improving the
reasoning capabilities of their language model component. Moreover, we explored
the ability of LVLMs to perceive image sequences within the context of our
multi-image association task. Our experiments show that the majority of current
LVLMs do not adequately model image sequences during the pre-training process.
