ISP-Agnostic Image Reconstruction for Under-Display Cameras

11/02/2021
by   Miao Qi, et al.
0

Under-display cameras have been proposed in recent years as a way to reduce the form factor of mobile devices while maximizing the screen area. Unfortunately, placing the camera behind the screen results in significant image distortions, including loss of contrast, blur, noise, color shift, scattering artifacts, and reduced light sensitivity. In this paper, we propose an image-restoration pipeline that is ISP-agnostic, i.e. it can be combined with any legacy ISP to produce a final image that matches the appearance of regular cameras using the same ISP. This is achieved with a deep learning approach that performs a RAW-to-RAW image restoration. To obtain large quantities of real under-display camera training data with sufficient contrast and scene diversity, we furthermore develop a data capture method utilizing an HDR monitor, as well as a data augmentation method to generate suitable HDR content. The monitor data is supplemented with real-world data that has less scene diversity but allows us to achieve fine detail recovery without being limited by the monitor resolution. Together, this approach successfully restores color and contrast as well as image detail.

READ FULL TEXT

page 17

page 18

page 20

page 22

page 23

page 24

page 33

page 35

research
03/10/2020

Image Restoration for Under-Display Camera

The new trend of full-screen devices encourages us to position a camera ...
research
05/20/2013

Efficient Image Retargeting for High Dynamic Range Scenes

Most of the real world scenes have a very high dynamic range (HDR). The ...
research
11/24/2022

Deep Demosaicing for Polarimetric Filter Array Cameras

Polarisation Filter Array (PFA) cameras allow the analysis of light pola...
research
03/20/2022

CRISPnet: Color Rendition ISP Net

Image signal processors (ISPs) are historically grown legacy software sy...
research
09/05/2022

UDC-UNet: Under-Display Camera Image Restoration via U-Shape Dynamic Network

Under-Display Camera (UDC) has been widely exploited to help smartphones...
research
04/06/2016

Reading Between the Pixels: Photographic Steganography for Camera Display Messaging

We exploit human color metamers to send light-modulated messages less vi...
research
08/25/2022

Fundamentals of Task-Agnostic Data Valuation

We study valuing the data of a data owner/seller for a data seeker/buyer...

Please sign up or login with your details

Forgot password? Click here to reset