Log In Sign Up

An Integrated Framework for the Heterogeneous Spatio-Spectral-Temporal Fusion of Remote Sensing Images

by   Menghui Jiang, et al.

Image fusion technology is widely used to fuse the complementary information between multi-source remote sensing images. Inspired by the frontier of deep learning, this paper first proposes a heterogeneous-integrated framework based on a novel deep residual cycle GAN. The proposed network consists of a forward fusion part and a backward degeneration feedback part. The forward part generates the desired fusion result from the various observations; the backward degeneration feedback part considers the imaging degradation process and regenerates the observations inversely from the fusion result. The proposed network can effectively fuse not only the homogeneous but also the heterogeneous information. In addition, for the first time, a heterogeneous-integrated fusion framework is proposed to simultaneously merge the complementary heterogeneous spatial, spectral and temporal information of multi-source heterogeneous observations. The proposed heterogeneous-integrated framework also provides a uniform mode that can complete various fusion tasks, including heterogeneous spatio-spectral fusion, spatio-temporal fusion, and heterogeneous spatio-spectral-temporal fusion. Experiments are conducted for two challenging scenarios of land cover changes and thick cloud coverage. Images from many remote sensing satellites, including MODIS, Landsat-8, Sentinel-1, and Sentinel-2, are utilized in the experiments. Both qualitative and quantitative evaluations confirm the effectiveness of the proposed method.


page 4

page 6

page 10

page 11

page 13

page 15

page 17


Correction of "Cloud Removal By Fusing Multi-Source and Multi-Temporal Images"

Remote sensing images often suffer from cloud cover. Cloud removal is re...

Graph-based fusion for change detection in multi-spectral images

In this paper we address the problem of change detection in multi-spectr...

MRFusion: A Deep Learning architecture to fuse PAN and MS imagery for land cover mapping

Nowadays, Earth Observation systems provide a multitude of heterogeneous...

Sentinel-1 and Sentinel-2 Spatio-Temporal Data Fusion for Clouds Removal

The abundance of clouds, located both spatially and temporally, often ma...

Multisource and Multitemporal Data Fusion in Remote Sensing

The sharp and recent increase in the availability of data captured by di...

A New Spatio-Spectral Morphological Segmentation For Multi-Spectral Remote-Sensing Images

A general framework of spatio-spectral segmentation for multi-spectral i...