  The area of portrait image animation, propelled by audio input, has witnessed
notable progress in the generation of lifelike and dynamic portraits.
Conventional methods are limited to utilizing either audios or facial key
points to drive images into videos, while they can yield satisfactory results,
certain issues exist. For instance, methods driven solely by audios can be
unstable at times due to the relatively weaker audio signal, while methods
driven exclusively by facial key points, although more stable in driving, can
result in unnatural outcomes due to the excessive control of key point
information. In addressing the previously mentioned challenges, in this paper,
we introduce a novel approach which we named EchoMimic. EchoMimic is
concurrently trained using both audios and facial landmarks. Through the
implementation of a novel training strategy, EchoMimic is capable of generating
portrait videos not only by audios and facial landmarks individually, but also
by a combination of both audios and selected facial landmarks. EchoMimic has
been comprehensively compared with alternative algorithms across various public
datasets and our collected dataset, showcasing superior performance in both
quantitative and qualitative evaluations. Additional visualization and access
to the source code can be located on the EchoMimic project page.
