A Cross-Modal Image Fusion Theory Guided by Human Visual Characteristics

12/18/2019
by   Aiqing Fang, et al.
15

The characteristics of feature selection, nonlinear combination and multi-task auxiliary learning mechanism of human visual perception system play an important role in real-world scenarios, but the research of image fusion theory based on the characteristics of human visual perception is less. Inspired by the characteristics of human visual perception, we propose a robust multi-task auxiliary learning optimization image fusion theory. Firstly, we combine channel attention model with nonlinear convolutional neural network to select features and fuse nonlinear features. Then, we analyze the impact of the existing image fusion loss on the image fusion quality, and establish the multi-loss function model of unsupervised learning network. Secondly, aiming at the multi-task auxiliary learning mechanism of human visual perception system, we study the influence of multi-task auxiliary learning mechanism on image fusion task on the basis of single task multi-loss network model. By simulating the three characteristics of human visual perception system, the fused image is more consistent with the mechanism of human brain image fusion. Finally, in order to verify the superiority of our algorithm, we carried out experiments on the combined vision system image data set, and extended our algorithm to the infrared and visible image and the multi-focus image public data set for experimental verification. The experimental results demonstrate the superiority of our fusion theory over state-of-arts in generality and robustness.

READ FULL TEXT

page 3

page 25

page 26

page 27

page 28

page 29

research
12/23/2019

Cross-Modal Image Fusion Theory Guided by Subjective Visual Attention

The human visual perception system has very strong robustness and contex...
research
12/23/2019

A Robust Non-Linear and Feature-Selection Image Fusion Theory

The human visual perception system has strong robustness in image fusion...
research
07/17/2020

AE-Net: Autonomous Evolution Image Fusion Method Inspired by Human Cognitive Mechanism

In order to solve the robustness and generality problems of the image fu...
research
11/11/2022

Palm Vein Recognition via Multi-task Loss Function and Attention Layer

With the improvement of arithmetic power and algorithm accuracy of perso...
research
10/05/2020

AE-Netv2: Optimization of Image Fusion Efficiency and Network Architecture

Existing image fusion methods pay few research attention to image fusion...
research
10/06/2019

Which Ads to Show? Advertisement Image Assessment with Auxiliary Information via Multi-step Modality Fusion

Assessing aesthetic preference is a fundamental task related to human co...
research
05/11/2023

Bi-level Dynamic Learning for Jointly Multi-modality Image Fusion and Beyond

Recently, multi-modality scene perception tasks, e.g., image fusion and ...

Please sign up or login with your details

Forgot password? Click here to reset