Facial recognition technology can now put your words into the mouth of anyone.
Software from Max Planck Institute for Informatics, University of Erlangen-Nuremberg, and Stanford University is able to create a real time reenactment of a person talking and map it onto another person's face. The result: you can make anyone say anything.
Dubbed Face2Face, the researchers combined an off-the-shelf webcam, facial tracking software, pre-existing video footage of the person they wanted to mimic and software to deform the original video. "Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion," the researchers wrote.
In a video published alongside their research the team showed the system reenacting YouTube videos in real-time. An actor speaks to the webcam and his facial expressions and speech are copied by George Bush, Vladimir Putin, Donald Trump, and Barack Obama.
The research, which is due to be presented at the IEEE Conference on Computer Vision and Pattern Recognition in June, works in real-time by tracking facial expressions of both the source and target video "using a dense photometric consistency measure".
Reenacting a face is made possible be "fast and efficient deformation transfer" between the source footage and the person being filmed. Mouth-shapes that best match what's being said are then warped to produce an accurate, believable fit.
As reported by Mashable, while just a research project for now the technology could be commercialised in the future.