Efficient Multi-Purpose Cross-Attention Based Image Alignment Block for Edge Devices

06/01/2022
by   Bahri Batuhan Bilecen, et al.
0

Image alignment, also known as image registration, is a critical block used in many computer vision problems. One of the key factors in alignment is efficiency, as inefficient aligners can cause significant overhead to the overall problem. In the literature, there are some blocks that appear to do the alignment operation, although most do not focus on efficiency. Therefore, an image alignment block which can both work in time and/or space and can work on edge devices would be beneficial for almost all networks dealing with multiple images. Given its wide usage and importance, we propose an efficient, cross-attention-based, multi-purpose image alignment block (XABA) suitable to work within edge devices. Using cross-attention, we exploit the relationships between features extracted from images. To make cross-attention feasible for real-time image alignment problems and handle large motions, we provide a pyramidal block based cross-attention scheme. This also captures local relationships besides reducing memory requirements and number of operations. Efficient XABA models achieve real-time requirements of running above 20 FPS performance on NVIDIA Jetson Xavier with 30W power consumption compared to other powerful computers. Used as a sub-block in a larger network, XABA also improves multi-image super-resolution network performance in comparison to other alignment methods.

READ FULL TEXT

page 1

page 6

page 7

page 8

research
07/17/2023

DARTS: Double Attention Reference-based Transformer for Super-resolution

We present DARTS, a transformer model for reference-based image super-re...
research
08/31/2021

Attention-based Multi-Reference Learning for Image Super-Resolution

This paper proposes a novel Attention-based Multi-Reference Super-resolu...
research
04/18/2022

Fast and Memory-Efficient Network Towards Efficient Image Super-Resolution

Runtime and memory consumption are two important aspects for efficient i...
research
12/17/2020

Attention-based Image Upsampling

Convolutional layers are an integral part of many deep neural network so...
research
07/18/2022

Rethinking Alignment in Video Super-Resolution Transformers

The alignment of adjacent frames is considered an essential operation in...
research
04/25/2022

IMDeception: Grouped Information Distilling Super-Resolution Network

Single-Image-Super-Resolution (SISR) is a classical computer vision prob...
research
04/02/2021

SDAN: Squared Deformable Alignment Network for Learning Misaligned Optical Zoom

Deep Neural Network (DNN) based super-resolution algorithms have greatly...

Please sign up or login with your details

Forgot password? Click here to reset