DeepAI AI Chat
Log In Sign Up

DiffFace: Diffusion-based Face Swapping with Facial Guidance

by   Kihong Kim, et al.

In this paper, we propose a diffusion-based face swapping framework for the first time, called DiffFace, composed of training ID conditional DDPM, sampling with facial guidance, and a target-preserving blending. In specific, in the training process, the ID conditional DDPM is trained to generate face images with the desired identity. In the sampling process, we use the off-the-shelf facial expert models to make the model transfer source identity while preserving target attributes faithfully. During this process, to preserve the background of the target image and obtain the desired face swapping result, we additionally propose a target-preserving blending strategy. It helps our model to keep the attributes of the target face from noise while transferring the source facial identity. In addition, without any re-training, our model can flexibly apply additional facial guidance and adaptively control the ID-attributes trade-off to achieve the desired results. To the best of our knowledge, this is the first approach that applies the diffusion model in face swapping task. Compared with previous GAN-based approaches, by taking advantage of the diffusion model for the face swapping task, DiffFace achieves better benefits such as training stability, high fidelity, diversity of the samples, and controllability. Extensive experiments show that our DiffFace is comparable or superior to the state-of-the-art methods on several standard face swapping benchmarks.


page 1

page 7

page 8

page 14

page 15

page 16

page 17

page 18


SimSwap: An Efficient Framework For High Fidelity Face Swapping

We propose an efficient framework, called Simple Swap (SimSwap), aiming ...

RobustSwap: A Simple yet Robust Face Swapping Model against Attribute Leakage

Face swapping aims at injecting a source image's identity (i.e., facial ...

Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection

Non-parametric face modeling aims to reconstruct 3D face only from image...

Investigating Prompt Engineering in Diffusion Models

With the spread of the use of Text2Img diffusion models such as DALL-E 2...

Instance-level Facial Attributes Transfer with Geometry-Aware Flow

We address the problem of instance-level facial attribute transfer witho...

FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping

In this work, we present a new single-stage method for subject agnostic ...

FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping

In this work, we propose a novel two-stage framework, called FaceShifter...