The Spatially-Correlative Loss for Various Image Translation Tasks

04/02/2021
by   Chuanxia Zheng, et al.
1

We propose a novel spatially-correlative loss that is simple, efficient and yet effective for preserving scene structure consistency while supporting large appearance changes during unpaired image-to-image (I2I) translation. Previous methods attempt this by using pixel-level cycle-consistency or feature-level matching losses, but the domain-specific nature of these losses hinder translation across large domain gaps. To address this, we exploit the spatial patterns of self-similarity as a means of defining scene structure. Our spatially-correlative loss is geared towards only capturing spatial relationships within an image rather than domain appearance. We also introduce a new self-supervised learning method to explicitly learn spatially-correlative maps for each specific translation task. We show distinct improvement over baseline models in all three modes of unpaired I2I translation: single-modal, multi-modal, and even single-image translation. This new loss can easily be integrated into existing network architectures and thus allows wide applicability.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 7

page 8

page 12

page 13

research
03/10/2020

Unpaired Image-to-Image Translation using Adversarial Consistency Loss

Unpaired image-to-image translation is a class of vision problems whose ...
research
11/29/2018

Image Translation to Mixed-Domain using Sym-Parameterized Generative Network

Recent advances in image-to-image translation have led to some ways to g...
research
03/03/2022

Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Tasks

Recently, contrastive learning-based image translation methods have been...
research
02/07/2019

Reversible GANs for Memory-efficient Image-to-Image Translation

The Pix2pix and CycleGAN losses have vastly improved the qualitative and...
research
05/29/2019

Image-to-Image Translation with Multi-Path Consistency Regularization

Image translation across different domains has attracted much attention ...
research
04/28/2020

Neural Hair Rendering

In this paper, we propose a generic neural-based hair rendering pipeline...
research
01/13/2020

Deep Image Translation with an Affinity-Based Change Prior for Unsupervised Multimodal Change Detection

Image translation with convolutional neural networks has recently been u...

Please sign up or login with your details

Forgot password? Click here to reset