Region-Aware Diffusion for Zero-shot Text-driven Image Editing

02/23/2023
by   Nisha Huang, et al.
0

Image manipulation under the guidance of textual descriptions has recently received a broad range of attention. In this study, we focus on the regional editing of images with the guidance of given text prompts. Different from current mask-based image editing methods, we propose a novel region-aware diffusion model (RDM) for entity-level image editing, which could automatically locate the region of interest and replace it following given text prompts. To strike a balance between image fidelity and inference speed, we design the intensive diffusion pipeline by combing latent space diffusion and enhanced directional guidance. In addition, to preserve image content in non-edited regions, we introduce regional-aware entity editing to modify the region of interest and preserve the out-of-interest region. We validate the proposed RDM beyond the baseline methods through extensive qualitative and quantitative experiments. The results show that RDM outperforms the previous approaches in terms of visual quality, overall harmonization, non-editing region content preservation, and text-image semantic consistency. The codes are available at https://github.com/haha-lisa/RDM-Region-Aware-Diffusion-Model.

READ FULL TEXT

page 1

page 4

page 5

page 7

page 10

page 11

page 12

page 15

research
07/05/2023

DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models

Despite the ability of existing large-scale text-to-image (T2I) models t...
research
12/05/2022

Fine-grained Image Editing by Pixel-wise Guidance Using Diffusion Models

Generative models, particularly GANs, have been utilized for image editi...
research
07/17/2023

Not All Steps are Created Equal: Selective Diffusion Distillation for Image Manipulation

Conditional diffusion models have demonstrated impressive performance in...
research
02/22/2023

Entity-Level Text-Guided Image Manipulation

Existing text-guided image manipulation methods aim to modify the appear...
research
06/21/2023

Local 3D Editing via 3D Distillation of CLIP Knowledge

3D content manipulation is an important computer vision task with many r...
research
11/29/2021

Blended Diffusion for Text-driven Editing of Natural Images

Natural language offers a highly intuitive interface for image editing. ...
research
06/07/2023

Designing a Better Asymmetric VQGAN for StableDiffusion

StableDiffusion is a revolutionary text-to-image generator that is causi...

Please sign up or login with your details

Forgot password? Click here to reset