SA-DNet: A on-demand semantic object registration network adapting to non-rigid deformation

10/18/2022
by   Housheng Xie, et al.
0

As an essential processing step before the fusing of infrared and visible images, the performance of image registration determines whether the two images can be fused at correct spatial position. In the actual scenario, the varied imaging devices may lead to a change in perspective or time gap between shots, making significant non-rigid spatial relationship in infrared and visible images. Even if a large number of feature points are matched, the registration accuracy may still be inadequate, affecting the result of image fusion and other vision tasks. To alleviate this problem, we propose a Semantic-Aware on-Demand registration network (SA-DNet), which mainly purpose is to confine the feature matching process to the semantic region of interest (sROI) by designing semantic-aware module (SAM) and HOL-Deep hybrid matching module (HDM). After utilizing TPS to transform infrared and visible images based on the corresponding feature points in sROI, the registered images are fused using image fusion module (IFM) to achieve a fully functional registration and fusion network. Moreover, we point out that for different demands, this type of approach allows us to select semantic objects for feature matching as needed and accomplishes task-specific registration based on specific requirements. To demonstrate the robustness of SA-DNet for non-rigid distortions, we conduct extensive experiments by comparing SA-DNet with five state-of-the-art infrared and visible image feature matching methods, and the experimental results show that our method adapts better to the presence of non-rigid distortions in the images and provides semantically well-registered images.

READ FULL TEXT

page 1

page 2

page 4

page 5

page 6

page 8

page 10

page 13

research
07/07/2023

Registration-Free Hybrid Learning Empowers Simple Multimodal Imaging System for High-quality Fusion Detection

Multimodal fusion detection always places high demands on the imaging sy...
research
02/21/2021

A Deep Decomposition Network for Image Processing: A Case Study for Visible and Infrared Image Fusion

Image decomposition is a crucial subject in the field of image processin...
research
05/24/2022

Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration

Recent learning-based image fusion methods have marked numerous progress...
research
04/12/2023

Modality-Invariant Representation for Infrared and Visible Image Registration

Since the differences in viewing range, resolution and relative position...
research
12/07/2018

SeFM: A Sequential Feature Point Matching Algorithm for Object 3D Reconstruction

3D reconstruction is a fundamental issue in many applications and the fe...
research
08/16/2022

A Hybrid Deep Feature-Based Deformable Image Registration Method for Pathological Images

Pathologists need to combine information from differently stained pathol...
research
09/15/2023

Viewpoint Integration and Registration with Vision Language Foundation Model for Image Change Understanding

Recently, the development of pre-trained vision language foundation mode...

Please sign up or login with your details

Forgot password? Click here to reset