Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN

by   Lloyd H. Hughes, et al.

In this letter, we propose a pseudo-siamese convolutional neural network (CNN) architecture that enables to solve the task of identifying corresponding patches in very-high-resolution (VHR) optical and synthetic aperture radar (SAR) remote sensing imagery. Using eight convolutional layers each in two parallel network streams, a fully connected layer for the fusion of the features learned in each stream, and a loss function based on binary cross-entropy, we achieve a one-hot indication if two patches correspond or not. The network is trained and tested on an automatically generated dataset that is based on a deterministic alignment of SAR and optical imagery via previously reconstructed and subsequently co-registered 3D point clouds. The satellite images, from which the patches comprising our dataset are extracted, show a complex urban scene containing many elevated objects (i.e. buildings), thus providing one of the most difficult experimental environments. The achieved results show that the network is able to predict corresponding patches with high accuracy, thus indicating great potential for further development towards a generalized multi-sensor key-point matching procedure. Index Terms-synthetic aperture radar (SAR), optical imagery, data fusion, deep learning, convolutional neural networks (CNN), image matching, deep matching


page 1

page 2

page 4

page 5


The QXS-SAROPT Dataset for Deep Learning in SAR-Optical Data Fusion

Deep learning techniques have made an increasing impact on the field of ...

CVPR MultiEarth 2023 Deforestation Estimation Challenge:SpaceVision4Amazon

In this paper, we present a deforestation estimation method based on att...

Road Segmentation in SAR Satellite Images with Deep Fully-Convolutional Neural Networks

Remote sensing is extensively used in cartography. As transportation net...

Vehicle Re-identification: exploring feature fusion using multi-stream convolutional networks

This work addresses the problem of vehicle re-identification through a n...

Optical Neural Networks

We develop a novel optical neural network (ONN) framework which introduc...

Buildings Detection in VHR SAR Images Using Fully Convolution Neural Networks

This paper addresses the highly challenging problem of automatically det...

Please sign up or login with your details

Forgot password? Click here to reset