Inversion-Based Style Transfer with Diffusion Models

11/23/2022
by   Yuxin Zhang, et al.
1

The artistic style within a painting is the means of expression, which includes not only the painting material, colors, and brushstrokes, but also the high-level attributes including semantic elements, object shapes, etc. Previous arbitrary example-guided artistic image generation methods often fail to control shape changes or convey elements. The pre-trained text-to-image synthesis diffusion probabilistic models have achieved remarkable quality, but it often requires extensive textual descriptions to accurately portray attributes of a particular painting. We believe that the uniqueness of an artwork lies precisely in the fact that it cannot be adequately explained with normal language. Our key idea is to learn artistic style directly from a single painting and then guide the synthesis without providing complex textual descriptions. Specifically, we assume style as a learnable textual description of a painting. We propose an inversion-based style transfer method (InST), which can efficiently and accurately learn the key information of an image, thus capturing and transferring the artistic style of a painting. We demonstrate the quality and efficiency of our method on numerous paintings of various artists and styles. Code and models are available at https://github.com/zyxElsa/InST.

READ FULL TEXT

page 1

page 2

page 4

page 6

page 7

page 8

research
08/29/2023

ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer

Textual style transfer is the task of transforming stylistic properties ...
research
06/01/2023

StyleDrop: Text-to-Image Generation in Any Style

Pre-trained large text-to-image models synthesize impressive images with...
research
03/06/2023

Neural Style Transfer for Vector Graphics

Neural style transfer draws researchers' attention, but the interest foc...
research
09/13/2023

DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models

Recent progresses in large-scale text-to-image models have yielded remar...
research
11/30/2022

Multiresolution Textual Inversion

We extend Textual Inversion to learn pseudo-words that represent a conce...
research
03/16/2023

P+: Extended Textual Conditioning in Text-to-Image Generation

We introduce an Extended Textual Conditioning space in text-to-image mod...
research
11/19/2022

DiffStyler: Controllable Dual Diffusion for Text-Driven Image Stylization

Despite the impressive results of arbitrary image-guided style transfer ...

Please sign up or login with your details

Forgot password? Click here to reset