DeepAI AI Chat
Log In Sign Up

TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network

01/25/2022
by   Dongyu Rao, et al.
Jiangnan University
NetEase, Inc
0

The end-to-end image fusion framework has achieved promising performance, with dedicated convolutional networks aggregating the multi-modal local appearance. However, long-range dependencies are directly neglected in existing CNN fusion approaches, impeding balancing the entire image-level perception for complex scenario fusion. In this paper, therefore, we propose an infrared and visible image fusion algorithm based on a lightweight transformer module and adversarial learning. Inspired by the global interaction power, we use the transformer technique to learn the effective global fusion relations. In particular, shallow features extracted by CNN are interacted in the proposed transformer fusion module to refine the fusion relationship within the spatial scope and across channels simultaneously. Besides, adversarial learning is designed in the training process to improve the output discrimination via imposing competitive consistency from the inputs, reflecting the specific characteristics in infrared and visible images. The experimental performance demonstrates the effectiveness of the proposed modules, with superior improvement against the state-of-the-art, generalising a novel paradigm via transformer and adversarial learning in the fusion task.

READ FULL TEXT

page 1

page 3

page 5

03/29/2022

Infrared and Visible Image Fusion via Interactive Compensatory Attention Adversarial Learning

The existing generative adversarial fusion methods generally concatenate...
04/25/2022

SwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images

The existing deep learning fusion methods mainly concentrate on the conv...
12/02/2021

TransMEF: A Transformer-Based Multi-Exposure Image Fusion Framework using Self-Supervised Multi-Task Learning

In this paper, we propose TransMEF, a transformer-based multi-exposure i...
01/24/2021

A Dual-branch Network for Infrared and Visible Image Fusion

Deep learning is a rapidly developing approach in the field of infrared ...
10/20/2022

An Attention-Guided and Wavelet-Constrained Generative Adversarial Network for Infrared and Visible Image Fusion

The GAN-based infrared and visible image fusion methods have gained ever...
06/20/2022

Explicit and implicit models in infrared and visible image fusion

Infrared and visible images, as multi-modal image pairs, show significan...