An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion

08/02/2022
by   Rinon Gal, et al.
2

Text-to-image models offer unprecedented freedom to guide creation through natural language. Yet, it is unclear how such freedom can be exercised to generate images of specific unique concepts, modify their appearance, or compose them in new roles and novel scenes. In other words, we ask: how can we use language-guided models to turn our cat into a painting, or imagine a new product based on our favorite toy? Here we present a simple approach that allows such creative freedom. Using only 3-5 images of a user-provided concept, like an object or a style, we learn to represent it through new "words" in the embedding space of a frozen text-to-image model. These "words" can be composed into natural language sentences, guiding personalized creation in an intuitive way. Notably, we find evidence that a single word embedding is sufficient for capturing unique and varied concepts. We compare our approach to a wide range of baselines, and demonstrate that it can more faithfully portray the concepts across a range of applications and tasks. Our code, data and new words will be available at: https://textual-inversion.github.io

READ FULL TEXT

page 6

page 7

page 8

page 11

page 22

page 23

page 24

page 25

research
08/21/2023

Backdooring Textual Inversion for Concept Censorship

Recent years have witnessed success in AIGC (AI Generated Content). Peop...
research
11/30/2022

Multiresolution Textual Inversion

We extend Textual Inversion to learn pseudo-words that represent a conce...
research
03/07/2023

ELODIN: Naming Concepts in Embedding Spaces

Despite recent advancements, the field of text-to-image synthesis still ...
research
02/07/2023

Concept Algebra for Text-Controlled Vision Models

This paper concerns the control of text-guided generative models, where ...
research
04/12/2023

Gradient-Free Textual Inversion

Recent works on personalized text-to-image generation usually learn to b...
research
03/16/2023

P+: Extended Textual Conditioning in Text-to-Image Generation

We introduce an Extended Textual Conditioning space in text-to-image mod...
research
08/04/2022

Adversarial Attacks on Image Generation With Made-Up Words

Text-guided image generation models can be prompted to generate images u...

Please sign up or login with your details

Forgot password? Click here to reset