Toward Foundation Models for Earth Monitoring: Generalizable Deep Learning Models for Natural Hazard Segmentation

by   Johannes Jakubik, et al.

Climate change results in an increased probability of extreme weather events that put societies and businesses at risk on a global scale. Therefore, near real-time mapping of natural hazards is an emerging priority for the support of natural disaster relief, risk management, and informing governmental policy decisions. Recent methods to achieve near real-time mapping increasingly leverage deep learning (DL). However, DL-based approaches are designed for one specific task in a single geographic region based on specific frequency bands of satellite data. Therefore, DL models used to map specific natural hazards struggle with their generalization to other types of natural hazards in unseen regions. In this work, we propose a methodology to significantly improve the generalizability of DL natural hazards mappers based on pre-training on a suitable pre-task. Without access to any data from the target domain, we demonstrate this improved generalizability across four U-Net architectures for the segmentation of unseen natural hazards. Importantly, our method is invariant to geographic differences and differences in the type of frequency bands of satellite data. By leveraging characteristics of unlabeled images from the target domain that are publicly available, our approach is able to further improve the generalization behavior without fine-tuning. Thereby, our approach supports the development of foundation models for earth monitoring with the objective of directly segmenting unseen natural hazards across novel geographic regions given different sources of satellite imagery.


page 1

page 2

page 3

page 4


EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts

Climate change is global, yet its concrete impacts can strongly vary bet...

MultiEarth 2022 – The Champion Solution for the Matrix Completion Challenge via Multimodal Regression and Generation

Earth observation satellites have been continuously monitoring the earth...

GRASP EARTH: Intuitive Software for Discovering Changes on the Planet

Detecting changes on the Earth, such as urban development, deforestation...

The data synergy effects of time-series deep learning models in hydrology

When fitting statistical models to variables in geoscientific discipline...

Deep Learning for Rain Fade Prediction in Satellite Communications

Line of sight satellite systems, unmanned aerial vehicles, high-altitude...

Single View Geocentric Pose in the Wild

Current methods for Earth observation tasks such as semantic mapping, ma...

Please sign up or login with your details

Forgot password? Click here to reset