Panoramic Image-to-Image Translation

04/11/2023
by   Soohyun Kim, et al.
0

In this paper, we tackle the challenging task of Panoramic Image-to-Image translation (Pano-I2I) for the first time. This task is difficult due to the geometric distortion of panoramic images and the lack of a panoramic image dataset with diverse conditions, like weather or time. To address these challenges, we propose a panoramic distortion-aware I2I model that preserves the structure of the panoramic images while consistently translating their global style referenced from a pinhole image. To mitigate the distortion issue in naive 360 panorama translation, we adopt spherical positional embedding to our transformer encoders, introduce a distortion-free discriminator, and apply sphere-based rotation for augmentation and its ensemble. We also design a content encoder and a style encoder to be deformation-aware to deal with a large domain gap between panoramas and pinhole images, enabling us to work on diverse conditions of pinhole images. In addition, considering the large discrepancy between panoramas and pinhole images, our framework decouples the learning procedure of the panoramic reconstruction stage from the translation stage. We show distinct improvements over existing I2I models in translating the StreetLearn dataset in the daytime into diverse conditions. The code will be publicly available online for our community.

READ FULL TEXT

page 7

page 8

page 15

page 17

page 20

page 21

page 22

page 23

research
07/15/2020

COCO-FUNIT: Few-Shot Unsupervised Image Translation with a Content Conditioned Style Encoder

Unsupervised image-to-image translation intends to learn a mapping of an...
research
05/05/2019

Towards Instance-level Image-to-Image Translation

Unpaired Image-to-image Translation is a new rising and challenging visi...
research
07/23/2019

Controlling biases and diversity in diverse image-to-image translation

The task of unpaired image-to-image translation is highly challenging du...
research
05/05/2021

Conditional Invertible Neural Networks for Diverse Image-to-Image Translation

We introduce a new architecture called a conditional invertible neural n...
research
11/11/2020

Generative and Discriminative Learning for Distorted Image Restoration

Liquify is a common technique for image editing, which can be used for i...
research
07/16/2023

Dense Multitask Learning to Reconfigure Comics

In this paper, we develop a MultiTask Learning (MTL) model to achieve de...
research
11/07/2018

DragonPaint: Rule based bootstrapping for small data with an application to cartoon coloring

In this paper, we confront the problem of deep learning's big labeled da...

Please sign up or login with your details

Forgot password? Click here to reset