  With the continuous development of Optical Character Recognition (OCR) and
the expansion of application fields, text recognition in complex scenes has
become a key challenge. Factors such as multiple fonts, mixed scenes and
complex layouts seriously affect the recognition accuracy of traditional OCR
models. Although OCR models based on deep learning have performed well in
specific fields or similar datasets in recent years, the generalization ability
and robustness of the model are still a big challenge when facing complex
environments with multiple scenes. Furthermore, training an OCR model from
scratch or fine-tuning all parameters is very demanding on computing resources
and inference time, which limits the flexibility of its application. This study
focuses on a fundamental aspect of mixed text recognition in response to the
challenges mentioned above, which involves effectively fine-tuning the
pre-trained basic OCR model to demonstrate exceptional performance across
various downstream tasks. To this end, we propose a parameter-efficient mixed
text recognition method based on pre-trained OCR Transformer, namely
DLoRA-TrOCR. This method embeds DoRA into the image encoder and LoRA into the
internal structure of the text decoder, enabling efficient parameter
fine-tuning for downstream tasks. Experiments show that compared to similar
parameter adjustment methods, our model DLoRA-TrOCR has the smallest number of
parameters and performs better. It can achieve state-of-the-art performance on
complex scene datasets involving simultaneous recognition of mixed handwritten,
printed and street view texts.
