HDR Reconstruction from Bracketed Exposures and Events

by   Richard Shaw, et al.

Reconstruction of high-quality HDR images is at the core of modern computational photography. Significant progress has been made with multi-frame HDR reconstruction methods, producing high-resolution, rich and accurate color reconstructions with high-frequency details. However, they are still prone to fail in dynamic or largely over-exposed scenes, where frame misalignment often results in visible ghosting artifacts. Recent approaches attempt to alleviate this by utilizing an event-based camera (EBC), which measures only binary changes of illuminations. Despite their desirable high temporal resolution and dynamic range characteristics, such approaches have not outperformed traditional multi-frame reconstruction methods, mainly due to the lack of color information and low-resolution sensors. In this paper, we propose to leverage both bracketed LDR images and simultaneously captured events to obtain the best of both worlds: high-quality RGB information from bracketed LDRs and complementary high frequency and dynamic range information from events. We present a multi-modal end-to-end learning-based HDR imaging system that fuses bracketed images and event modalities in the feature domain using attention and multi-scale spatial alignment modules. We propose a novel event-to-image feature distillation module that learns to translate event features into the image-feature space with self-supervision. Our framework exploits the higher temporal resolution of events by sub-sampling the input event streams using a sliding window, enriching our combined feature representation. Our proposed approach surpasses SoTA multi-frame HDR reconstruction methods using synthetic and real events, with a 2dB and 1dB improvement in PSNR-L and PSNR-mu on the HdM HDR dataset, respectively.


page 2

page 11

page 13

page 22

page 23

page 24

page 25

page 26


Multi-Bracket High Dynamic Range Imaging with Event Cameras

Modern high dynamic range (HDR) imaging pipelines align and fuse multipl...

Event-based Image Deblurring with Dynamic Motion Awareness

Non-uniform image deblurring is a challenging task due to the lack of te...

LAN-HDR: Luminance-based Alignment Network for High Dynamic Range Video Reconstruction

As demands for high-quality videos continue to rise, high-resolution and...

Information-Preserved Blending Method for Forward-Looking Sonar Mosaicing in Non-Ideal System Configuration

Forward-Looking Sonar (FLS) has started to gain attention in the field o...

Event Enhanced High-Quality Image Recovery

With extremely high temporal resolution, event cameras have a large pote...

A Dynamic Graph CNN with Cross-Representation Distillation for Event-Based Recognition

It is a popular solution to convert events into dense frame-based repres...

Injecting Image Details into CLIP's Feature Space

Although CLIP-like Visual Language Models provide a functional joint fea...

Please sign up or login with your details

Forgot password? Click here to reset