MD-CSDNetwork: Multi-Domain Cross Stitched Network for Deepfake Detection

by   Aayushi Agarwal, et al.

The rapid progress in the ease of creating and spreading ultra-realistic media over social platforms calls for an urgent need to develop a generalizable deepfake detection technique. It has been observed that current deepfake generation methods leave discriminative artifacts in the frequency spectrum of fake images and videos. Inspired by this observation, in this paper, we present a novel approach, termed as MD-CSDNetwork, for combining the features in the spatial and frequency domains to mine a shared discriminative representation for classifying deepfakes. MD-CSDNetwork is a novel cross-stitched network with two parallel branches carrying the spatial and frequency information, respectively. We hypothesize that these multi-domain input data streams can be considered as related supervisory signals. The supervision from both branches ensures better performance and generalization. Further, the concept of cross-stitch connections is utilized where they are inserted between the two branches to learn an optimal combination of domain-specific and shared representations from other domains automatically. Extensive experiments are conducted on the popular benchmark dataset namely FaceForeniscs++ for forgery classification. We report improvements over all the manipulation types in FaceForensics++ dataset and comparable results with state-of-the-art methods for cross-database evaluation on the Celeb-DF dataset and the Deepfake Detection Dataset.


page 1

page 2

page 4

page 5


Spatial-Temporal Frequency Forgery Clue for Video Forgery Detection in VIS and NIR Scenario

In recent years, with the rapid development of face editing and generati...

Fake Visual Content Detection Using Two-Stream Convolutional Neural Networks

Rapid progress in adversarial learning has enabled the generation of rea...

MC-LCR: Multi-modal contrastive classification by locally correlated representations for effective face forgery detection

As the remarkable development of facial manipulation technologies is acc...

Deepfake Detection via Joint Unsupervised Reconstruction and Supervised Classification

Deep learning has enabled realistic face manipulation (i.e., deepfake), ...

Detecting Deepfakes with Self-Blended Images

In this paper, we present novel synthetic training data called self-blen...

Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain

The remarkable success in face forgery techniques has received considera...

Collaborative Three-Stream Transformers for Video Captioning

As the most critical components in a sentence, subject, predicate and ob...

Please sign up or login with your details

Forgot password? Click here to reset