HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery

02/18/2020
by   Alfredo Kalaitzis, et al.
1

Generative deep learning has sparked a new wave of Super-Resolution (SR) algorithms that enhance single images with impressive aesthetic results, albeit with imaginary details. Multi-frame Super-Resolution (MFSR) offers a more grounded approach to the ill-posed problem, by conditioning on multiple low-resolution views. This is important for satellite monitoring of human impact on the planet -- from deforestation, to human rights violations -- that depend on reliable imagery. To this end, we present HighRes-net, the first deep learning approach to MFSR that learns its sub-tasks in an end-to-end fashion: (i) co-registration, (ii) fusion, (iii) up-sampling, and (iv) registration-at-the-loss. Co-registration of low-resolution views is learned implicitly through a reference-frame channel, with no explicit registration mechanism. We learn a global fusion operator that is applied recursively on an arbitrary number of low-resolution pairs. We introduce a registered loss, by learning to align the SR output to a ground-truth through ShiftNet. We show that by learning deep representations of multiple views, we can super-resolve low-resolution signals and enhance Earth Observation data at scale. Our approach recently topped the European Space Agency's MFSR competition on real-world satellite imagery.

READ FULL TEXT

page 2

page 4

research
07/15/2019

DeepSUM: Deep neural network for Super-resolution of Unregistered Multitemporal images

Recently, convolutional neural networks (CNN) have been successfully app...
research
08/03/2020

Fusion of Deep and Non-Deep Methods for Fast Super-Resolution of Satellite Images

In the emerging commercial space industry there is a drastic increase in...
research
03/10/2023

Handheld Burst Super-Resolution Meets Multi-Exposure Satellite Imagery

Image resolution is an important criterion for many applications based o...
research
12/05/2022

Region-Conditioned Orthogonal 3D U-Net for Weather4Cast Competition

The Weather4Cast competition (hosted by NeurIPS 2022) required competito...
research
09/08/2017

Benchmarking Super-Resolution Algorithms on Real Data

Over the past decades, various super-resolution (SR) techniques have bee...
research
07/13/2022

Open High-Resolution Satellite Imagery: The WorldStrat Dataset – With Application to Super-Resolution

Analyzing the planet at scale with satellite imagery and machine learnin...
research
07/03/2019

Super-Resolution of PROBA-V Images Using Convolutional Neural Networks

ESA's PROBA-V Earth observation satellite enables us to monitor our plan...

Please sign up or login with your details

Forgot password? Click here to reset