  Automatic lip-reading (ALR) aims to automatically transcribe spoken content
from a speaker's silent lip motion captured in video. Current mainstream
lip-reading approaches only use a single visual encoder to model input videos
of a single scale. In this paper, we propose to enhance lip-reading by
incorporating multi-scale video data and multi-encoder. Specifically, we first
propose a novel multi-scale lip motion extraction algorithm based on the size
of the speaker's face and an Enhanced ResNet3D visual front-end (VFE) to
extract lip features at different scales. For the multi-encoder, in addition to
the mainstream Transformer and Conformer, we also incorporate the recently
proposed Branchformer and E-Branchformer as visual encoders. In the
experiments, we explore the influence of different video data scales and
encoders on ALR system performance and fuse the texts transcribed by all ALR
systems using recognizer output voting error reduction (ROVER). Finally, our
proposed approach placed second in the ICME 2024 ChatCLR Challenge Task 2, with
a 21.52% reduction in character error rate (CER) compared to the official
baseline on the evaluation set.
