Dynamic Instance Normalization for Arbitrary Style Transfer

11/16/2019
by   Yongcheng Jing, et al.
0

Prior normalization methods rely on affine transformations to produce arbitrary image style transfers, of which the parameters are computed in a pre-defined way. Such manually-defined nature eventually results in the high-cost and shared encoders for both style and content encoding, making style transfer systems cumbersome to be deployed in resource-constrained environments like on the mobile-terminal side. In this paper, we propose a new and generalized normalization module, termed as Dynamic Instance Normalization (DIN), that allows for flexible and more efficient arbitrary style transfers. Comprising an instance normalization and a dynamic convolution, DIN encodes a style image into learnable convolution parameters, upon which the content image is stylized. Unlike conventional methods that use shared complex encoders to encode content and style, the proposed DIN introduces a sophisticated style encoder, yet comes with a compact and lightweight content encoder for fast inference. Experimental results demonstrate that the proposed approach yields very encouraging results on challenging style patterns and, to our best knowledge, for the first time enables an arbitrary style transfer using MobileNet-based lightweight architecture, leading to a reduction factor of more than twenty in computational cost as compared to existing approaches. Furthermore, the proposed DIN provides flexible support for state-of-the-art convolutional operations, and thus triggers novel functionalities, such as uniform-stroke placement for non-natural images and automatic spatial-stroke control.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

research
10/06/2020

Arbitrary Style Transfer using Graph Instance Normalization

Style transfer is the image synthesis task, which applies a style of one...
research
03/20/2017

Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization

Gatys et al. recently introduced a neural algorithm that renders a conte...
research
11/28/2022

MicroAST: Towards Super-Fast Ultra-Resolution Arbitrary Style Transfer

Arbitrary style transfer (AST) transfers arbitrary artistic styles onto ...
research
05/13/2021

SAFIN: Arbitrary Style Transfer With Self-Attentive Factorized Instance Normalization

Artistic style transfer aims to transfer the style characteristics of on...
research
04/02/2023

Learning Dynamic Style Kernels for Artistic Style Transfer

Arbitrary style transfer has been demonstrated to be efficient in artist...
research
01/11/2021

Generalize Ultrasound Image Segmentation via Instant and Plug Play Style Transfer

Deep segmentation models that generalize to images with unknown appearan...
research
09/12/2021

CropDefender: deep watermark which is more convenient to train and more robust against cropping

Digital image watermarking, which is a technique for invisibly embedding...

Please sign up or login with your details

Forgot password? Click here to reset