CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks

06/23/2020
by   Mahmoud Afifi, et al.
23

Cameras currently allow access to two image states: (i) a minimally processed linear raw-RGB image state (i.e., raw sensor data) or (ii) a highly-processed nonlinear image state (e.g., sRGB). There are many computer vision tasks that work best with a linear image state, such as image deblurring and image dehazing. Unfortunately, the vast majority of images are saved in the nonlinear image state. Because of this, a number of methods have been proposed to "unprocess" nonlinear images back to a raw-RGB state. However, existing unprocessing methods have a drawback because raw-RGB images are sensor-specific. As a result, it is necessary to know which camera produced the sRGB output and use a method or network tailored for that sensor to properly unprocess it. This paper addresses this limitation by exploiting another camera image state that is not available as an output, but it is available inside the camera pipeline. In particular, cameras apply a colorimetric conversion step to convert the raw-RGB image to a device-independent space based on the CIE XYZ color space before they apply the nonlinear photo-finishing. Leveraging this canonical image state, we propose a deep learning framework, CIE XYZ Net, that can unprocess a nonlinear image back to the canonical CIE XYZ image. This image can then be processed by any low-level computer vision operator and re-rendered back to the nonlinear image. We demonstrate the usefulness of the CIE XYZ Net on several low-level vision tasks and show significant gains that can be obtained by this processing framework. Code and dataset are publicly available at https://github.com/mahmoudnafifi/CIE_XYZ_NET.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 7

page 10

page 13

research
12/15/2022

Efficient Visual Computing with Camera RAW Snapshots

Conventional cameras capture image irradiance on a sensor and convert it...
research
07/26/2017

Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network

We present a novel deep learning framework that models the scene depende...
research
01/25/2021

ISP Distillation

Nowadays, many of the images captured are "observed" by machines only an...
research
12/14/2019

Sensor-Independent Illumination Estimation for DNN Models

While modern deep neural networks (DNNs) achieve state-of-the-art result...
research
10/11/2022

Enabling ISP-less Low-Power Computer Vision

In order to deploy current computer vision (CV) models on resource-const...
research
12/16/2021

All You Need is RAW: Defending Against Adversarial Attacks with Camera Image Pipelines

Existing neural networks for computer vision tasks are vulnerable to adv...
research
10/20/2022

Overexposure Mask Fusion: Generalizable Reverse ISP Multi-Step Refinement

With the advent of deep learning methods replacing the ISP in transformi...

Please sign up or login with your details

Forgot password? Click here to reset