Deepfake detection: humans vs. machines

09/07/2020
by   Pavel Korshunov, et al.
21

Deepfake videos, where a person's face is automatically swapped with a face of someone else, are becoming easier to generate with more realistic results. In response to the threat such manipulations can pose to our trust in video evidence, several large datasets of deepfake videos and many methods to detect them were proposed recently. However, it is still unclear how realistic deepfake videos are for an average person and whether the algorithms are significantly better than humans at detecting them. In this paper, we present a subjective study conducted in a crowdsourcing-like scenario, which systematically evaluates how hard it is for humans to see if the video is deepfake or not. For the evaluation, we used 120 different videos (60 deepfakes and 60 originals) manually pre-selected from the Facebook deepfake database, which was provided in the Kaggle's Deepfake Detection Challenge 2020. For each video, a simple question: "Is face of the person in the video real of fake?" was answered on average by 19 naïve subjects. The results of the subjective evaluation were compared with the performance of two different state of the art deepfake detection methods, based on Xception and EfficientNets (B4 variant) neural networks, which were pre-trained on two other large public databases: the Google's subset from FaceForensics++ and the recent Celeb-DF dataset. The evaluation demonstrates that while the human perception is very different from the perception of a machine, both successfully but in different ways are fooled by deepfakes. Specifically, algorithms struggle to detect those deepfake videos, which human subjects found to be very easy to spot.

READ FULL TEXT

page 1

page 2

page 3

page 5

research
12/20/2018

DeepFakes: a New Threat to Face Recognition? Assessment and Detection

It is becoming increasingly easy to automatically replace a face of one ...
research
09/07/2021

Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors

Significant advancements made in the generation of deepfakes have caused...
research
09/10/2019

Swapped Face Detection using Deep Learning and Subjective Assessment

The tremendous success of deep learning for imaging applications has res...
research
06/12/2020

The DeepFake Detection Challenge Dataset

Deepfakes are a recent off-the-shelf manipulation technique that allows ...
research
05/02/2019

Recurrent-Convolution Approach to DeepFake Detection - State-Of-Art Results on FaceForensics++

Spread of misinformation has become a significant problem, raising the i...
research
09/25/2018

Covfefe: A Computer Vision Approach For Estimating Force Exertion

Cumulative exposure to repetitive and forceful activities may lead to mu...
research
06/01/2022

Deepfake Caricatures: Amplifying attention to artifacts increases deepfake detection by humans and machines

Deepfakes pose a serious threat to our digital society by fueling the sp...

Please sign up or login with your details

Forgot password? Click here to reset