  Existing methods for long video understanding primarily focus on videos only
lasting tens of seconds, with limited exploration of techniques for handling
longer videos. The increased number of frames in longer videos presents two
main challenges: difficulty in locating key information and performing
long-range reasoning. Thus, we propose DrVideo, a document-retrieval-based
system designed for long video understanding. Our key idea is to convert the
long-video understanding problem into a long-document understanding task so as
to effectively leverage the power of large language models. Specifically,
DrVideo transforms a long video into a text-based long document to initially
retrieve key frames and augment the information of these frames, which is used
this as the system's starting point. It then employs an agent-based iterative
loop to continuously search for missing information, augment relevant data, and
provide final predictions in a chain-of-thought manner once sufficient
question-related information is gathered. Extensive experiments on long video
benchmarks confirm the effectiveness of our method. DrVideo outperforms
existing state-of-the-art methods with +3.8 accuracy on EgoSchema benchmark (3
minutes), +17.9 in MovieChat-1K break mode, +38.0 in MovieChat-1K global mode
(10 minutes), and +30.2 on the LLama-Vid QA dataset (over 60 minutes).
