DeepAI AI Chat
Log In Sign Up

ManiCLIP: Multi-Attribute Face Manipulation from Text

10/02/2022
by   Hao Wang, et al.
0

In this paper we present a novel multi-attribute face manipulation method based on textual descriptions. Previous text-based image editing methods either require test-time optimization for each individual image or are restricted to single attribute editing. Extending these methods to multi-attribute face image editing scenarios will introduce undesired excessive attribute change, e.g., text-relevant attributes are overly manipulated and text-irrelevant attributes are also changed. In order to address these challenges and achieve natural editing over multiple face attributes, we propose a new decoupling training scheme where we use group sampling to get text segments from same attribute categories, instead of whole complex sentences. Further, to preserve other existing face attributes, we encourage the model to edit the latent code of each attribute separately via a entropy constraint. During the inference phase, our model is able to edit new face images without any test-time optimization, even from complex textual prompts. We show extensive experiments and analysis to demonstrate the efficacy of our method, which generates natural manipulated faces with minimal text-irrelevant attribute editing. Code and pre-trained model will be released.

READ FULL TEXT

page 1

page 2

page 4

page 6

page 7

page 8

07/20/2018

Editable Generative Adversarial Networks: Generating and Editing Faces Simultaneously

We propose a novel framework for simultaneously generating and manipulat...
11/29/2017

Arbitrary Facial Attribute Editing: Only Change What You Want

Facial attribute editing aims to modify either single or multiple attrib...
07/17/2023

CLIP-Guided StyleGAN Inversion for Text-Driven Real Image Editing

Researchers have recently begun exploring the use of StyleGAN-based mode...
10/21/2021

Each Attribute Matters: Contrastive Attention for Sentence-based Image Editing

Sentence-based Image Editing (SIE) aims to deploy natural language to ed...
10/12/2022

Leveraging Off-the-shelf Diffusion Model for Multi-attribute Fashion Image Manipulation

Fashion attribute editing is a task that aims to convert the semantic at...
12/16/2016

Learning Residual Images for Face Attribute Manipulation

Face attributes are interesting due to their detailed description of hum...