Visual Realism Assessment for Face-swap Videos

by   Xianyun Sun, et al.

Deep-learning based face-swap videos, also known as deep fakes, are becoming more and more realistic and deceiving. The malicious usage of these face-swap videos has caused wide concerns. The research community has been focusing on the automatic detection of these fake videos, but the as sessment of their visual realism, as perceived by human eyes, is still an unexplored dimension. Visual realism assessment, or VRA, is essential for assessing the potential impact that may be brought by a specific face-swap video, and it is also important as a quality assessment metric to compare different face-swap methods. In this paper, we make a small step to wards this new VRA direction by building a benchmark for evaluating the effectiveness of different automatic VRA models, which range from using traditional hand-crafted features to different kinds of deep-learning features. The evaluations are based on a recent competition dataset named as DFGC 2022, which contains 1400 diverse face-swap videos that are annotated with Mean Opinion Scores (MOS) on visual realism. Comprehensive experiment results using 11 models and 3 protocols are shown and discussed. We demonstrate the feasibility of devising effective VRA models for assessing face-swap videos and methods. The particular usefulness of existing deepfake detection features for VRA is also noted. The code and benchmark will be made publicly available.


VideoForensicsHQ: Detecting High-quality Manipulated Face Videos

New approaches to synthesize and manipulate face videos at very high qua...

DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection

In this paper, we present our on-going effort of constructing a large-sc...

DeepFakes: a New Threat to Face Recognition? Assessment and Detection

It is becoming increasingly easy to automatically replace a face of one ...

A Survey of Hand Crafted and Deep Learning Methods for Image Aesthetic Assessment

Automatic image aesthetics assessment is a computer vision problem that ...

Swapped Face Detection using Deep Learning and Subjective Assessment

The tremendous success of deep learning for imaging applications has res...

The Making and Breaking of Camouflage

Not all camouflages are equally effective, as even a partially visible c...

GAN-based Deidentification of Drivers' Face Videos: An Assessment of Human Factors Implications in NDS Data

This paper addresses the problem of sharing drivers' face videos for tra...

Please sign up or login with your details

Forgot password? Click here to reset