An international team of experts has created a new method capable of identifying videos manipulated by artificial intelligence, known as deepfakes, achieving an average accuracy rate exceeding 95%.
New Detection Approach
This methodology, developed by researchers from the University of Tokyo in Japan and the Max Planck Institute for Informatics in Germany, innovates by not focusing on visual flaws. Instead, the system analyzes whether the facial expressions presented correspond naturally to the audio spoken by the person.
The researchers report that this system managed to detect manipulations that escaped existing detectors, marking a significant advance in creating more efficient tools against falsified content.
Deepfake Context
Experts point out that generative AI already produces images and videos almost indistinguishable from real recordings to the human eye. Although this technology has beneficial uses, it also increases the risks of fraud, identity theft, and the spread of disinformation, making the development of reliable identification methods a priority in AI research.
Limitations of Previous Methods
The authors explain that current most accurate deepfake detectors generally use supervised learning, trained with vast databases of authentic and altered videos. However, this model can suffer from overfitting, learning specific characteristics of certain falsification methods and losing efficiency against new techniques.
In contrast, unsupervised methods are trained only with true videos, making them more resistant to new technologies but typically showing lower accuracy. The new proposed technique is the first unsupervised approach that manages to combine a high detection rate with robustness, surpassing current supervised methods.
Facial Analysis Versus Audio
Instead of looking for inconsistencies in pixels or other visual artifacts, the system concentrates its analysis on natural facial movements. It uses the FLAME model, common in facial animation and computer graphics, which describes facial expressions through 53 mathematical parameters.
During development, the researchers pre-trained the model with over 450 hours of public videos. With this material, the model learned to anticipate which facial movements would naturally be expected from a specific soundtrack. After this initial training, the system can be personalized for an individual using only about 60 seconds of video, functioning as a personalized deepfake detector.
In the analysis phase, the software compares the facial movements observed in the video with those predicted based on the speech. Noticeable differences between these two datasets are interpreted as strong signs of manipulation. Vladislav Golyanik stated that combining unsupervised learning with FLAME-based facial analysis gives the approach special resistance against new deepfake generation methods, as well as distortions such as noise or image compression.
Results in Rigorous Tests
In tests conducted by the team, the method achieved an average accuracy above 95% across several scientific reference datasets, surpassing the performance of state-of-the-art techniques. A considerable challenge was the evaluation on a dataset created by the researchers themselves, containing videos generated by Sora 2, OpenAI's video generation tool.
While previous detectors showed results close to chance, the new system correctly identified almost 95% of the manipulated videos. Despite these advances, the researchers warn that the technology still has limitations, as it requires extensive pre-training on powerful hardware and currently does not operate in real time.
