AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer

08/08/2021
by   Songhua Liu, et al.
16

Fast arbitrary neural style transfer has attracted widespread attention from academic, industrial and art communities due to its flexibility in enabling various applications. Existing solutions either attentively fuse deep style feature into deep content feature without considering feature distributions, or adaptively normalize deep content feature according to the style such that their global statistics are matched. Although effective, leaving shallow feature unexplored and without locally considering feature statistics, they are prone to unnatural output with unpleasing local distortions. To alleviate this problem, in this paper, we propose a novel attention and normalization module, named Adaptive Attention Normalization (AdaAttN), to adaptively perform attentive normalization on per-point basis. Specifically, spatial attention score is learnt from both shallow and deep features of content and style images. Then per-point weighted statistics are calculated by regarding a style feature point as a distribution of attention-weighted output of all style feature points. Finally, the content feature is normalized so that they demonstrate the same local feature statistics as the calculated per-point weighted style feature statistics. Besides, a novel local feature loss is derived based on AdaAttN to enhance local visual quality. We also extend AdaAttN to be ready for video style transfer with slight modifications. Experiments demonstrate that our method achieves state-of-the-art arbitrary image/video style transfer. Codes and models are available.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

page 12

page 13

page 14

research
12/08/2022

All-to-key Attention for Arbitrary Style Transfer

Attention-based arbitrary style transfer studies have shown promising pe...
research
05/27/2020

Arbitrary Style Transfer via Multi-Adaptation Network

Arbitrary style transfer is a significant topic with both research value...
research
01/06/2022

Consistent Style Transfer

Recently, attentional arbitrary style transfer methods have been propose...
research
08/13/2021

UMFA: A photorealistic style transfer method based on U-Net and multi-layer feature aggregation

In this paper, we propose a photorealistic style transfer network to emp...
research
04/02/2023

Learning Dynamic Style Kernels for Artistic Style Transfer

Arbitrary style transfer has been demonstrated to be efficient in artist...
research
05/13/2021

SAFIN: Arbitrary Style Transfer With Self-Attentive Factorized Instance Normalization

Artistic style transfer aims to transfer the style characteristics of on...
research
03/15/2022

Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalization

Arbitrary style transfer (AST) and domain generalization (DG) are import...

Please sign up or login with your details

Forgot password? Click here to reset