Representation Learning for Remote Sensing: An Unsupervised Sensor Fusion Approach

08/11/2021
by   Aidan M. Swope, et al.
11

In the application of machine learning to remote sensing, labeled data is often scarce or expensive, which impedes the training of powerful models like deep convolutional neural networks. Although unlabeled data is abundant, recent self-supervised learning approaches are ill-suited to the remote sensing domain. In addition, most remote sensing applications currently use only a small subset of the multi-sensor, multi-channel information available, motivating the need for fused multi-sensor representations. We propose a new self-supervised training objective, Contrastive Sensor Fusion, which exploits coterminous data from multiple sources to learn useful representations of every possible combination of those sources. This method uses information common across multiple sensors and bands by training a single model to produce a representation that remains similar when any subset of its input channels is used. Using a dataset of 47 million unlabeled coterminous image triplets, we train an encoder to produce semantically meaningful representations from any possible combination of channels from the input sensors. These representations outperform fully supervised ImageNet weights on a remote sensing classification task and improve as more sensors are fused. Our code is available at https://storage.cloud.google.com/public-published-datasets/csf_code.zip.

READ FULL TEXT

page 2

page 4

page 8

research
01/29/2023

Supervised and Contrastive Self-Supervised In-Domain Representation Learning for Dense Prediction Problems in Remote Sensing

In recent years Convolutional neural networks (CNN) have made significan...
research
11/16/2021

SEnSeI: A Deep Learning Module for Creating Sensor Independent Cloud Masks

We introduce a novel neural network architecture – Spectral ENcoder for ...
research
12/28/2016

MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification

With the development of deep learning, supervised learning has frequentl...
research
08/20/2013

Influences Combination of Multi-Sensor Images on Classification Accuracy

This paper focuses on two main issues; first one is the impact of combin...
research
08/16/2021

Clustering augmented Self-Supervised Learning: Anapplication to Land Cover Mapping

Collecting large annotated datasets in Remote Sensing is often expensive...
research
06/16/2023

Joint multi-modal Self-Supervised pre-training in Remote Sensing: Application to Methane Source Classification

With the current ubiquity of deep learning methods to solve computer vis...
research
10/19/2022

p^3VAE: a physics-integrated generative model. Application to the semantic segmentation of optical remote sensing images

The combination of machine learning models with physical models is a rec...

Please sign up or login with your details

Forgot password? Click here to reset