Ghost-free High Dynamic Range Imaging with Context-aware Transformer

08/10/2022
by   Zhen Liu, et al.
0

High dynamic range (HDR) deghosting algorithms aim to generate ghost-free HDR images with realistic details. Restricted by the locality of the receptive field, existing CNN-based methods are typically prone to producing ghosting artifacts and intensity distortions in the presence of large motion and severe saturation. In this paper, we propose a novel Context-Aware Vision Transformer (CA-ViT) for ghost-free high dynamic range imaging. The CA-ViT is designed as a dual-branch architecture, which can jointly capture both global and local dependencies. Specifically, the global branch employs a window-based Transformer encoder to model long-range object movements and intensity variations to solve ghosting. For the local branch, we design a local context extractor (LCE) to capture short-range image features and use the channel attention mechanism to select informative local details across the extracted features to complement the global branch. By incorporating the CA-ViT as basic components, we further build the HDR-Transformer, a hierarchical network to reconstruct high-quality ghost-free HDR images. Extensive experiments on three benchmark datasets show that our approach outperforms state-of-the-art methods qualitatively and quantitatively with considerably reduced computational budgets. Codes are available at https://github.com/megvii-research/HDR-Transformer

READ FULL TEXT

page 2

page 11

page 12

research
04/10/2023

High Dynamic Range Imaging with Context-aware Transformer

Avoiding the introduction of ghosts when synthesising LDR images as high...
research
07/19/2021

Image Fusion Transformer

In image fusion, images obtained from different sensors are fused to gen...
research
12/01/2022

Ghost-free High Dynamic Range Imaging via Hybrid CNN-Transformer and Structure Tensor

Eliminating ghosting artifacts due to moving objects is a challenging pr...
research
05/29/2023

Alignment-free HDR Deghosting with Semantics Consistent Transformer

High dynamic range (HDR) imaging aims to retrieve information from multi...
research
05/23/2022

SelfReformer: Self-Refined Network with Transformer for Salient Object Detection

The global and local contexts significantly contribute to the integrity ...
research
04/14/2022

3D Shuffle-Mixer: An Efficient Context-Aware Vision Learner of Transformer-MLP Paradigm for Dense Prediction in Medical Volume

Dense prediction in medical volume provides enriched guidance for clinic...
research
09/19/2023

Context-Aware Neural Video Compression on Solar Dynamics Observatory

NASA's Solar Dynamics Observatory (SDO) mission collects large data volu...

Please sign up or login with your details

Forgot password? Click here to reset