A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning

03/09/2023
by   Yuxin Zhang, et al.
0

We present Unified Contrastive Arbitrary Style Transfer (UCAST), a novel style representation learning and transfer framework, which can fit in most existing arbitrary image style transfer models, e.g., CNN-based, ViT-based, and flow-based methods. As the key component in image style transfer tasks, a suitable style representation is essential to achieve satisfactory results. Existing approaches based on deep neural network typically use second-order statistics to generate the output. However, these hand-crafted features computed from a single image cannot leverage style information sufficiently, which leads to artifacts such as local distortions and style inconsistency. To address these issues, we propose to learn style representation directly from a large amount of images based on contrastive learning, by taking the relationships between specific styles and the holistic style distribution into account. Specifically, we present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer. We carry out qualitative and quantitative evaluations to show that our approach produces superior results than those obtained via state-of-the-art methods.

READ FULL TEXT
research
05/19/2022

Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning

In this work, we tackle the challenging problem of arbitrary image style...
research
05/10/2018

Arbitrary Style Transfer with Deep Feature Reshuffle

This paper introduces a novel method by reshuffling deep features (i.e.,...
research
02/21/2022

MIST GAN: Modality Imputation Using Style Transfer for MRI

MRI entails a great amount of cost, time and effort for the generation o...
research
09/03/2021

3D Human Shape Style Transfer

We consider the problem of modifying/replacing the shape style of a real...
research
03/16/2023

SpectralCLIP: Preventing Artifacts in Text-Guided Style Transfer from a Spectral Perspective

Contrastive Language-Image Pre-Training (CLIP) has refreshed the state o...
research
12/24/2022

Meta-Learning for Color-to-Infrared Cross-Modal Style Transfer

Recent object detection models for infrared (IR) imagery are based upon ...
research
04/15/2022

Patch-wise Contrastive Style Learning for Instagram Filter Removal

Image-level corruptions and perturbations degrade the performance of CNN...

Please sign up or login with your details

Forgot password? Click here to reset