  Despite significant breakthroughs in video analysis driven by the rapid
development of large multimodal models (LMMs), there remains a lack of a
versatile evaluation benchmark to comprehensively assess these models'
performance in video understanding and reasoning. To address this, we present
VideoVista, a video QA benchmark that integrates challenges across diverse
content categories, durations, and abilities. Specifically, VideoVista
comprises 25,000 questions derived from 3,400 videos spanning 14 categories
(e.g., Howto, Film, and Entertainment) with durations ranging from a few
seconds to over 10 minutes. Besides, it encompasses 19 types of understanding
tasks (e.g., anomaly detection, interaction understanding) and 8 reasoning
tasks (e.g., logical reasoning, causal reasoning). To achieve this, we present
an automatic data construction framework, leveraging powerful GPT-4o alongside
advanced analysis tools (e.g., video splitting, object segmenting, and
tracking). We also utilize this framework to construct training data to enhance
the capabilities of video-related LMMs (Video-LMMs). Through a comprehensive
and quantitative evaluation of cutting-edge models, we reveal that: 1)
Video-LMMs face difficulties in fine-grained video tasks involving temporal
location, object tracking, and anomaly detection; 2) Video-LMMs present
inferior logical and relation reasoning abilities; 3) Open-source Video-LMMs'
performance is significantly lower than GPT-4o and Gemini-1.5, lagging by 20
points. This highlights the crucial role VideoVista will play in advancing LMMs
that can accurately understand videos and perform precise reasoning.
