  High-resolution inputs enable Large Vision-Language Models (LVLMs) to discern
finer visual details, enhancing their comprehension capabilities. To reduce the
training and computation costs caused by high-resolution input, one promising
direction is to use sliding windows to slice the input into uniform patches,
each matching the input size of the well-trained vision encoder. Although
efficient, this slicing strategy leads to the fragmentation of original input,
i.e., the continuity of contextual information and spatial geometry is lost
across patches, adversely affecting performance in cross-patch context
perception and position-specific tasks. To overcome these shortcomings, we
introduce HiRes-LLaVA, a novel framework designed to efficiently process any
size of high-resolution input without altering the original contextual and
geometric information. HiRes-LLaVA comprises two innovative components: (i) a
SliceRestore adapter that reconstructs sliced patches into their original form,
efficiently extracting both global and local features via down-up-sampling and
convolution layers, and (ii) a Self-Mining Sampler to compresses the vision
tokens based on themselves, preserving the original context and positional
information while reducing training overhead. To assess the ability of handling
context fragmentation, we construct a new benchmark, EntityGrid-QA, consisting
of edge-related and position-related tasks. Our comprehensive experiments
demonstrate the superiority of HiRes-LLaVA on both existing public benchmarks
and on EntityGrid-QA, particularly on document-oriented tasks, establishing new
standards for handling high-resolution inputs.
