ISP-Agnostic Image Reconstruction for Under-Display Cameras

by   Miao Qi, et al.

Under-display cameras have been proposed in recent years as a way to reduce the form factor of mobile devices while maximizing the screen area. Unfortunately, placing the camera behind the screen results in significant image distortions, including loss of contrast, blur, noise, color shift, scattering artifacts, and reduced light sensitivity. In this paper, we propose an image-restoration pipeline that is ISP-agnostic, i.e. it can be combined with any legacy ISP to produce a final image that matches the appearance of regular cameras using the same ISP. This is achieved with a deep learning approach that performs a RAW-to-RAW image restoration. To obtain large quantities of real under-display camera training data with sufficient contrast and scene diversity, we furthermore develop a data capture method utilizing an HDR monitor, as well as a data augmentation method to generate suitable HDR content. The monitor data is supplemented with real-world data that has less scene diversity but allows us to achieve fine detail recovery without being limited by the monitor resolution. Together, this approach successfully restores color and contrast as well as image detail.


page 17

page 18

page 20

page 22

page 23

page 24

page 33

page 35


Image Restoration for Under-Display Camera

The new trend of full-screen devices encourages us to position a camera ...

Efficient Image Retargeting for High Dynamic Range Scenes

Most of the real world scenes have a very high dynamic range (HDR). The ...

Deep Demosaicing for Polarimetric Filter Array Cameras

Polarisation Filter Array (PFA) cameras allow the analysis of light pola...

CRISPnet: Color Rendition ISP Net

Image signal processors (ISPs) are historically grown legacy software sy...

UDC-UNet: Under-Display Camera Image Restoration via U-Shape Dynamic Network

Under-Display Camera (UDC) has been widely exploited to help smartphones...

Reading Between the Pixels: Photographic Steganography for Camera Display Messaging

We exploit human color metamers to send light-modulated messages less vi...

Fundamentals of Task-Agnostic Data Valuation

We study valuing the data of a data owner/seller for a data seeker/buyer...

Please sign up or login with your details

Forgot password? Click here to reset