Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Tasks

03/03/2022
by   Chanyong Jung, et al.
0

Recently, contrastive learning-based image translation methods have been proposed, which contrasts different spatial locations to enhance the spatial correspondence. However, the methods often ignore the diverse semantic relation within the images. To address this, here we propose a novel semantic relation consistency (SRC) regularization along with the decoupled contrastive learning, which utilize the diverse semantics by focusing on the heterogeneous semantics between the image patches of a single image. To further improve the performance, we present a hard negative mining by exploiting the semantic relation. We verified our method for three tasks: single-modal and multi-modal image translations, and GAN compression task for image translation. Experimental results confirmed the state-of-art performance of our method in all the three tasks.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

page 8

research
08/10/2021

Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation

Contrastive learning shows great potential in unpaired image-to-image tr...
research
04/25/2023

Unsupervised Synthetic Image Refinement via Contrastive Learning and Consistent Semantic-Structural Constraints

Ensuring the realism of computer-generated synthetic images is crucial t...
research
02/21/2023

Unpaired Translation from Semantic Label Maps to Images by Leveraging Domain-Specific Simulations

Photorealistic image generation from simulated label maps are necessitat...
research
09/06/2023

Exploring Semantic Consistency in Unpaired Image Translation to Generate Data for Surgical Applications

In surgical computer vision applications, obtaining labeled training dat...
research
04/02/2021

The Spatially-Correlative Loss for Various Image Translation Tasks

We propose a novel spatially-correlative loss that is simple, efficient ...
research
07/09/2023

Latent Graph Attention for Enhanced Spatial Context

Global contexts in images are quite valuable in image-to-image translati...
research
04/22/2023

Spectral normalized dual contrastive regularization for image-to-image translation

Existing image-to-image(I2I) translation methods achieve state-of-the-ar...

Please sign up or login with your details

Forgot password? Click here to reset