PCSGAN: Perceptual Cyclic-Synthesized Generative Adversarial Networks for Thermal and NIR to Visible Image Transformation

In many real world scenarios, it is difficult to capture the images in the visible light spectrum (VIS) due to bad lighting conditions. However, the images can be captured in such scenarios using Near-Infrared (NIR) and Thermal (THM) cameras. The NIR and THM images contain the limited details. Thus, there is a need to transform the images from THM/NIR to VIS for better understanding. However, it is non-trivial task due to the large domain discrepancies and lack of abundant datasets. Nowadays, Generative Adversarial Network (GAN) is able to transform the images from one domain to another domain. Most of the available GAN based methods use the combination of the adversarial and the pixel-wise losses (like L1 or L2) as the objective function for training. The quality of transformed images in case of THM/NIR to VIS transformation is still not up to the mark using such objective function. Thus, better objective functions are needed to improve the quality, fine details and realism of the transformed images. A new model for THM/NIR to VIS image transformation called Perceptual Cyclic-Synthesized Generative Adversarial Network (PCSGAN) is introduced to address these issues. The PCSGAN uses the combination of the perceptual (i.e., feature based) losses along with the pixel-wise and the adversarial losses. Both the quantitative and qualitative measures are used to judge the performance of the PCSGAN model over the WHU-IIP face and the RGB-NIR scene datasets. The proposed PCSGAN outperforms the state-of-the-art image transformation models, including Pix2pix, DualGAN, CycleGAN, PS2GAN, and PAN in terms of the SSIM, MSE, PSNR and LPIPS evaluation measures. The code is available at: <https://github.com/KishanKancharagunta/PCSGAN>.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 8

page 9

research
01/11/2019

CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation

The primary motivation of Image-to-Image Transformation is to convert an...
research
10/12/2018

Thermal Infrared Colorization via Conditional Generative Adversarial Network

Transforming a thermal infrared image into a realistic RGB image is a ch...
research
01/15/2020

CDGAN: Cyclic Discriminative Generative Adversarial Networks for Image-to-Image Transformation

Image-to-image transformation is a kind of problem, where the input imag...
research
04/03/2019

Unpaired Thermal to Visible Spectrum Transfer using Adversarial Training

Thermal Infrared (TIR) cameras are gaining popularity in many computer v...
research
12/12/2018

Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

The large domain discrepancy between faces captured in polarimetric (or ...
research
11/26/2017

Semantically Consistent Image Completion with Fine-grained Details

Image completion has achieved significant progress due to advances in ge...
research
02/14/2019

GeoGAN: A Conditional GAN with Reconstruction and Style Loss to Generate Standard Layer of Maps from Satellite Images

Automatically generating maps from satellite images is an important task...

Please sign up or login with your details

Forgot password? Click here to reset