Day-to-Night Image Synthesis for Training Nighttime Neural ISPs

06/06/2022
by   Abhijith Punnappurath, et al.
0

Many flagship smartphone cameras now use a dedicated neural image signal processor (ISP) to render noisy raw sensor images to the final processed output. Training nightmode ISP networks relies on large-scale datasets of image pairs with: (1) a noisy raw image captured with a short exposure and a high ISO gain; and (2) a ground truth low-noise raw image captured with a long exposure and low ISO that has been rendered through the ISP. Capturing such image pairs is tedious and time-consuming, requiring careful setup to ensure alignment between the image pairs. In addition, ground truth images are often prone to motion blur due to the long exposure. To address this problem, we propose a method that synthesizes nighttime images from daytime images. Daytime images are easy to capture, exhibit low-noise (even on smartphone cameras) and rarely suffer from motion blur. We outline a processing framework to convert daytime raw images to have the appearance of realistic nighttime raw images with different levels of noise. Our procedure allows us to easily produce aligned noisy and clean nighttime image pairs. We show the effectiveness of our synthesis framework by training neural ISPs for nightmode rendering. Furthermore, we demonstrate that using our synthetic nighttime images together with small amounts of real data (e.g., 5 par with training exclusively on real nighttime images. Our dataset and code are available at https://github.com/SamsungLabs/day-to-night.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 7

page 10

page 11

research
11/23/2018

LSD_2 - Joint Denoising and Deblurring of Short and Long Exposure Images with Convolutional Neural Networks

This paper addresses the challenging problem of acquiring high-quality p...
research
07/07/2022

D2HNet: Joint Denoising and Deblurring with Hierarchical Network for Robust Night Image Restoration

Night imaging with modern smartphone cameras is troublesome due to low p...
research
11/26/2021

NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images

Neural Radiance Fields (NeRF) is a technique for high quality novel view...
research
08/07/2023

Lighting Every Darkness in Two Pairs: A Calibration-Free Pipeline for RAW Denoising

Calibration-based methods have dominated RAW image denoising under extre...
research
07/05/2017

Benchmarking Denoising Algorithms with Real Photographs

Lacking realistic ground truth data, image denoising techniques are trad...
research
01/27/2021

DeepOIS: Gyroscope-Guided Deep Optical Image Stabilizer Compensation

Mobile captured images can be aligned using their gyroscope sensors. Opt...
research
12/22/2020

HDR Denoising and Deblurring by Learning Spatio-temporal Distortion Models

We seek to reconstruct sharp and noise-free high-dynamic range (HDR) vid...

Please sign up or login with your details

Forgot password? Click here to reset