Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Models

by   Jun Yue, et al.

Color plays an important role in human visual perception, reflecting the spectrum of objects. However, the existing infrared and visible image fusion methods rarely explore how to handle multi-spectral/channel data directly and achieve high color fidelity. This paper addresses the above issue by proposing a novel method with diffusion models, termed as Dif-Fusion, to generate the distribution of the multi-channel input data, which increases the ability of multi-source information aggregation and the fidelity of colors. In specific, instead of converting multi-channel images into single-channel data in existing fusion methods, we create the multi-channel data distribution with a denoising network in a latent space with forward and reverse diffusion process. Then, we use the the denoising network to extract the multi-channel diffusion features with both visible and infrared information. Finally, we feed the multi-channel diffusion features to the multi-channel fusion module to directly generate the three-channel fused image. To retain the texture and intensity information, we propose multi-channel gradient loss and intensity loss. Along with the current evaluation metrics for measuring texture and intensity fidelity, we introduce a new evaluation metric to quantify color fidelity. Extensive experiments indicate that our method is more effective than other state-of-the-art image fusion methods, especially in color fidelity.


page 1

page 4

page 5

page 7

page 8

page 9

page 10

page 13


NestFuse: An Infrared and Visible Image Fusion Architecture based on Nest Connection and Spatial/Channel Attention Models

In this paper we propose a novel method for infrared and visible image f...

DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion

Multi-modality image fusion aims to combine different modalities to prod...

Infrared and Visible Image Fusion via Interactive Compensatory Attention Adversarial Learning

The existing generative adversarial fusion methods generally concatenate...

Visible and NIR Image Fusion Algorithm Based on Information Complementarity

Visible and near-infrared(NIR) band sensors provide images that capture ...

Joint Intensity-Gradient Guided Generative Modeling for Colorization

This paper proposes an iterative generative model for solving the automa...

Visible and infrared self-supervised fusion trained on a single example

This paper addresses the problem of visible (RGB) to Near-Infrared (NIR)...

Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models

Public large-scale text-to-image diffusion models, such as Stable Diffus...

Please sign up or login with your details

Forgot password? Click here to reset