Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets

by   Ke Li, et al.

This work studies Hyperspectral image (HSI) super-resolution (SR). HSI SR is characterized by high-dimensional data and a limited amount of training examples. This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples. This work addresses these issues with three contributions. First, we propose a simple, yet effective data augmentation routine, termed Spectral Mixup, to construct effective virtual training samples. Second, we observe that HSI SR and RGB image SR are correlated and develop a novel multi-tasking network to train them jointly so that the auxiliary task RGB image SR can provide additional supervision. Finally, we extend the network to a semi-supervised setting so that it can learn from datasets containing low-resolution HSIs only. With these contributions, our method is able to learn from heterogeneous datasets and lift the requirement for having a large amount of HD HSI training samples. Extensive experiments on four datasets show that our method outperforms existing methods significantly and underpin the relevance of our contributions. The code of this work will be released soon.


page 4

page 7

page 14

page 15


Spatial-Spectral Feedback Network for Super-Resolution of Hyperspectral Imagery

Recently, single gray/RGB image super-resolution (SR) methods based on d...

HST: Hierarchical Swin Transformer for Compressed Image Super-resolution

Compressed Image Super-resolution has achieved great attention in recent...

A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution

Realistic hyperspectral image (HSI) super-resolution (SR) techniques aim...

Hyperspectral Image Super-resolution via Deep Spatio-spectral Convolutional Neural Networks

Hyperspectral images are of crucial importance in order to better unders...

Unsupervised and Unregistered Hyperspectral Image Super-Resolution with Mutual Dirichlet-Net

Hyperspectral images (HSI) provide rich spectral information that contri...

Physically Plausible Spectral Reconstruction from RGB Images

Recently Convolutional Neural Networks (CNN) have been used to reconstru...

Suppressing Model Overfitting for Image Super-Resolution Networks

Large deep networks have demonstrated competitive performance in single ...

Code Repositories