Lightweight HDR Camera ISP for Robust Perception in Dynamic Illumination Conditions via Fourier Adversarial Networks

04/04/2022
by   Pranjay Shyam, et al.
0

The limited dynamic range of commercial compact camera sensors results in an inaccurate representation of scenes with varying illumination conditions, adversely affecting image quality and subsequently limiting the performance of underlying image processing algorithms. Current state-of-the-art (SoTA) convolutional neural networks (CNN) are developed as post-processing techniques to independently recover under-/over-exposed images. However, when applied to images containing real-world degradations such as glare, high-beam, color bleeding with varying noise intensity, these algorithms amplify the degradations, further degrading image quality. We propose a lightweight two-stage image enhancement algorithm sequentially balancing illumination and noise removal using frequency priors for structural guidance to overcome these limitations. Furthermore, to ensure realistic image quality, we leverage the relationship between frequency and spatial domain properties of an image and propose a Fourier spectrum-based adversarial framework (AFNet) for consistent image enhancement under varying illumination conditions. While current formulations of image enhancement are envisioned as post-processing techniques, we examine if such an algorithm could be extended to integrate the functionality of the Image Signal Processing (ISP) pipeline within the camera sensor benefiting from RAW sensor data and lightweight CNN architecture. Based on quantitative and qualitative evaluations, we also examine the practicality and effects of image enhancement techniques on the performance of common perception tasks such as object detection and semantic segmentation in varying illumination conditions.

READ FULL TEXT

page 10

page 13

page 14

page 16

page 18

page 19

page 21

page 22

research
08/03/2021

Del-Net: A Single-Stage Network for Mobile Camera ISP

The quality of images captured by smartphones is an important specificat...
research
04/22/2020

Learning an Adaptive Model for Extreme Low-light Raw Image Processing

Low-light images suffer from severe noise and low illumination. Current ...
research
05/06/2007

Enhancement of Noisy Planar Nuclear Medicine Images using Mean Field Annealing

Nuclear medicine (NM) images inherently suffer from large amounts of noi...
research
08/05/2019

CameraNet: A Two-Stage Framework for Effective Camera ISP Learning

Traditional image signal processing (ISP) pipeline consists of a set of ...
research
07/30/2021

Exploring Low-light Object Detection Techniques

Images acquired by computer vision systems under low light conditions ha...
research
02/26/2022

DGSS : Domain Generalized Semantic Segmentation using Iterative Style Mining and Latent Representation Alignment

Semantic segmentation algorithms require access to well-annotated datase...
research
10/28/2022

Boulders Identification on Small Bodies Under Varying Illumination Conditions

The capability to detect boulders on the surface of small bodies is bene...

Please sign up or login with your details

Forgot password? Click here to reset