  Creating realistic, natural, and lip-readable talking face videos remains a
formidable challenge. Previous research primarily concentrated on generating
and aligning single-frame images while overlooking the smoothness of
frame-to-frame transitions and temporal dependencies. This often compromised
visual quality and effects in practical settings, particularly when handling
complex facial data and audio content, which frequently led to semantically
incongruent visual illusions. Specifically, synthesized videos commonly
featured disorganized lip movements, making them difficult to understand and
recognize. To overcome these limitations, this paper introduces the application
of optical flow to guide facial image generation, enhancing inter-frame
continuity and semantic consistency. We propose "OpFlowTalker", a novel
approach that utilizes predicted optical flow changes from audio inputs rather
than direct image predictions. This method smooths image transitions and aligns
changes with semantic content. Moreover, it employs a sequence fusion technique
to replace the independent generation of single frames, thus preserving
contextual information and maintaining temporal coherence. We also developed an
optical flow synchronization module that regulates both full-face and lip
movements, optimizing visual synthesis by balancing regional dynamics.
Furthermore, we introduce a Visual Text Consistency Score (VTCS) that
accurately measures lip-readability in synthesized videos. Extensive empirical
evidence validates the effectiveness of our approach.
