Investigating Prompt Engineering in Diffusion Models

by   Sam Witteveen, et al.

With the spread of the use of Text2Img diffusion models such as DALL-E 2, Imagen, Mid Journey and Stable Diffusion, one challenge that artists face is selecting the right prompts to achieve the desired artistic output. We present techniques for measuring the effect that specific words and phrases in prompts have, and (in the Appendix) present guidance on the selection of prompts to produce desired effects.


page 5

page 6

page 7

page 8

page 10


Universal Guidance for Diffusion Models

Typical diffusion models are trained to accept a particular form of cond...

DiffFace: Diffusion-based Face Swapping with Facial Guidance

In this paper, we propose a diffusion-based face swapping framework for ...

Drag-guided diffusion models for vehicle image generation

Denoising diffusion models trained at web-scale have revolutionized imag...

Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models

Cutting-edge diffusion models produce images with high quality and custo...

Directed Diffusion: Direct Control of Object Placement through Attention Guidance

Text-guided diffusion models such as DALLE-2, IMAGEN, and Stable Diffusi...

Diffusion Models for Computational Design at the Example of Floor Plans

AI Image generators based on diffusion models are widely discussed recen...

Imitating Human Behaviour with Diffusion Models

Diffusion models have emerged as powerful generative models in the text-...

Please sign up or login with your details

Forgot password? Click here to reset