Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution

07/10/2020
by   Jing Yao, et al.
0

The recent advancement of deep learning techniques has made great progress on hyperspectral image super-resolution (HSI-SR). Yet the development of unsupervised deep networks remains challenging for this task. To this end, we propose a novel coupled unmixing network with a cross-attention mechanism, CUCaNet for short, to enhance the spatial resolution of HSI by means of higher-spatial-resolution multispectral image (MSI). Inspired by coupled spectral unmixing, a two-stream convolutional autoencoder framework is taken as backbone to jointly decompose MS and HS data into a spectrally meaningful basis and corresponding coefficients. CUCaNet is capable of adaptively learning spectral and spatial response functions from HS-MS correspondences by enforcing reasonable consistency assumptions on the networks. Moreover, a cross-attention module is devised to yield more effective spatial-spectral information transfer in networks. Extensive experiments are conducted on three widely-used HS-MS datasets in comparison with state-of-the-art HSI-SR models, demonstrating the superiority of the CUCaNet in the HSI-SR application. Furthermore, the codes and datasets will be available at: https://github.com/danfenghong/ECCV2020_CUCaNet.

READ FULL TEXT

page 4

page 5

page 11

page 12

page 13

page 14

research
05/18/2020

Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery

Recently, single gray/RGB image super-resolution reconstruction task has...
research
07/28/2020

Spectral Superresolution of Multispectral Imagery with Joint Sparse and Low-Rank Learning

Extensive attention has been widely paid to enhance the spatial resoluti...
research
01/14/2022

Unsupervised Data Fusion with Deeper Perspective: A Novel Multi-Sensor Deep Clustering Algorithm

The ever-growing developments in technology to capture different types o...
research
03/04/2022

HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening

Pansharpening aims to fuse a registered high-resolution panchromatic ima...
research
07/28/2020

Coupled Convolutional Neural Network with Adaptive Response Function Learning for Unsupervised Hyperspectral Super-Resolution

Due to the limitations of hyperspectral imaging systems, hyperspectral i...
research
01/19/2021

Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets

This work studies Hyperspectral image (HSI) super-resolution (SR). HSI S...
research
04/27/2019

Unsupervised and Unregistered Hyperspectral Image Super-Resolution with Mutual Dirichlet-Net

Hyperspectral images (HSI) provide rich spectral information that contri...

Please sign up or login with your details

Forgot password? Click here to reset