Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network

04/19/2021
by   Ruicheng Feng, et al.
17

Recent development of Under-Display Camera (UDC) systems provides a true bezel-less and notch-free viewing experience on smartphones (and TV, laptops, tablets), while allowing images to be captured from the selfie camera embedded underneath. In a typical UDC system, the microstructure of the semi-transparent organic light-emitting diode (OLED) pixel array attenuates and diffracts the incident light on the camera, resulting in significant image quality degradation. Oftentimes, noise, flare, haze, and blur can be observed in UDC images. In this work, we aim to analyze and tackle the aforementioned degradation problems. We define a physics-based image formation model to better understand the degradation. In addition, we utilize one of the world's first commodity UDC smartphone prototypes to measure the real-world Point Spread Function (PSF) of the UDC system, and provide a model-based data synthesis pipeline to generate realistically degraded images. We specially design a new domain knowledge-enabled Dynamic Skip Connection Network (DISCNet) to restore the UDC images. We demonstrate the effectiveness of our method through extensive experiments on both synthetic and real UDC data. Our physics-based image formation model and proposed DISCNet can provide foundations for further exploration in UDC image restoration, and even for general diffraction artifact removal in a broader sense.

READ FULL TEXT

page 1

page 8

page 12

page 15

page 16

page 17

page 18

page 19

research
03/10/2020

Image Restoration for Under-Display Camera

The new trend of full-screen devices encourages us to position a camera ...
research
09/05/2022

UDC-UNet: Under-Display Camera Image Restoration via U-Shape Dynamic Network

Under-Display Camera (UDC) has been widely exploited to help smartphones...
research
08/08/2023

Under-Display Camera Image Restoration with Scattering Effect

The under-display camera (UDC) provides consumers with a full-screen vis...
research
06/07/2023

Flare7K++: Mixing Synthetic and Real Datasets for Nighttime Flare Removal and Beyond

Artificial lights commonly leave strong lens flare artifacts on the imag...
research
08/01/2020

Joint Generative Learning and Super-Resolution For Real-World Camera-Screen Degradation

In real-world single image super-resolution (SISR) task, the low-resolut...
research
11/23/2022

BAD-NeRF: Bundle Adjusted Deblur Neural Radiance Fields

Neural Radiance Fields (NeRF) have received considerable attention recen...
research
01/08/2015

Optimal Radiometric Calibration for Camera-Display Communication

We present a novel method for communicating between a camera and display...

Please sign up or login with your details

Forgot password? Click here to reset