Self-supervised HDR Imaging from Motion and Exposure Cues

by   Michal Nazarczuk, et al.

Recent High Dynamic Range (HDR) techniques extend the capabilities of current cameras where scenes with a wide range of illumination can not be accurately captured with a single low-dynamic-range (LDR) image. This is generally accomplished by capturing several LDR images with varying exposure values whose information is then incorporated into a merged HDR image. While such approaches work well for static scenes, dynamic scenes pose several challenges, mostly related to the difficulty of finding reliable pixel correspondences. Data-driven approaches tackle the problem by learning an end-to-end mapping with paired LDR-HDR training data, but in practice generating such HDR ground-truth labels for dynamic scenes is time-consuming and requires complex procedures that assume control of certain dynamic elements of the scene (e.g. actor pose) and repeatable lighting conditions (stop-motion capturing). In this work, we propose a novel self-supervised approach for learnable HDR estimation that alleviates the need for HDR ground-truth labels. We propose to leverage the internal statistics of LDR images to create HDR pseudo-labels. We separately exploit static and well-exposed parts of the input images, which in conjunction with synthetic illumination clipping and motion augmentation provide high quality training examples. Experimental results show that the HDR models trained using our proposed self-supervision approach achieve performance competitive with those trained under full supervision, and are to a large extent superior to previous methods that equally do not require any supervision.


page 2

page 7

page 12

page 16

page 17

page 18

page 19

page 21


Self-supervised Outdoor Scene Relighting

Outdoor scene relighting is a challenging problem that requires good und...

Learnable Exposure Fusion for Dynamic Scenes

In this paper, we focus on Exposure Fusion (EF) [ExposFusi2] for dynamic...

SMAE: Few-shot Learning for HDR Deghosting with Saturation-Aware Masked Autoencoders

Generating a high-quality High Dynamic Range (HDR) image from dynamic sc...

Motion Degeneracy in Self-supervised Learning of Elevation Angle Estimation for 2D Forward-Looking Sonar

2D forward-looking sonar is a crucial sensor for underwater robotic perc...

Geometric Consistency for Self-Supervised End-to-End Visual Odometry

With the success of deep learning based approaches in tackling challengi...

DeepLight: Learning Illumination for Unconstrained Mobile Mixed Reality

We present a learning-based method to infer plausible high dynamic range...

Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding

For many fundamental scene understanding tasks, it is difficult or impos...

Please sign up or login with your details

Forgot password? Click here to reset