DiffusionCLIP: Text-guided Image Manipulation Using Diffusion Models

10/06/2021
by   Gwanghyun Kim, et al.
0

Diffusion models are recent generative models that have shown great success in image generation with the state-of-the-art performance. However, only a few researches have been conducted for image manipulation with diffusion models. Here, we present a novel DiffusionCLIP which performs text-driven image manipulation with diffusion models using Contrastive Language-Image Pre-training (CLIP) loss. Our method has a performance comparable to that of the modern GAN-based image processing methods for in and out-of-domain image processing tasks, with the advantage of almost perfect inversion even without additional encoders or optimization. Furthermore, our method can be easily used for various novel applications, enabling image translation from an unseen domain to another unseen domain or stroke-conditioned image generation in an unseen domain, etc. Finally, we present a novel multiple attribute control with DiffusionCLIPby combining multiple fine-tuned diffusion models.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

page 9

research
10/05/2022

LDEdit: Towards Generalized Text Guided Image Manipulation via Latent Diffusion Models

Research in vision-language models has seen rapid developments off-late,...
research
06/07/2023

Improving Diffusion-based Image Translation using Asymmetric Gradient Guidance

Diffusion models have shown significant progress in image translation ta...
research
12/05/2022

CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphics

Considerable progress has recently been made in leveraging CLIP (Contras...
research
04/02/2023

Textile Pattern Generation Using Diffusion Models

The problem of text-guided image generation is a complex task in Compute...
research
09/05/2023

Gradient Domain Diffusion Models for Image Synthesis

Diffusion models are getting popular in generative image and video synth...
research
08/03/2021

Toward Spatially Unbiased Generative Models

Recent image generation models show remarkable generation performance. H...
research
05/26/2023

High-Fidelity Image Compression with Score-based Generative Models

Despite the tremendous success of diffusion generative models in text-to...

Please sign up or login with your details

Forgot password? Click here to reset