  Cropping high-resolution document images into multiple sub-images is the most
widely used approach for current Multimodal Large Language Models (MLLMs) to do
document understanding. Most of current document understanding methods preserve
all tokens within sub-images and treat them equally. This neglects their
different informativeness and leads to a significant increase in the number of
image tokens. To perform a more adaptive and efficient document understanding,
we propose Token-level Correlation-guided Compression, a parameter-free and
plug-and-play methodology to optimize token processing. Firstly, we propose an
innovative approach for assessing the pattern repetitiveness based on the
correlation between each patch tokens. This method identifies redundant tokens,
allowing for the determination of the sub-image's information density.
Secondly, we present a token-level sampling method that efficiently captures
the most informative tokens by delving into the correlation between the [CLS]
token and patch tokens. By integrating these strategies, we develop a
plug-and-play adaptive compressor module that can be seamlessly incorporated
into MLLMs utilizing cropping techniques. This module not only enhances the
processing speed during training and inference but also maintains comparable
performance. We conduct experiments with the SOTA document understanding model
mPLUG-DocOwl1.5 and the effectiveness is demonstrated through extensive
comparisons with other compression methods.
