Dirty Pixels: Optimizing Image Classification Architectures for Raw Sensor Data

01/23/2017
by   Steven Diamond, et al.
0

Real-world sensors suffer from noise, blur, and other imperfections that make high-level computer vision tasks like scene segmentation, tracking, and scene understanding difficult. Making high-level computer vision networks robust is imperative for real-world applications like autonomous driving, robotics, and surveillance. We propose a novel end-to-end differentiable architecture for joint denoising, deblurring, and classification that makes classification robust to realistic noise and blur. The proposed architecture dramatically improves the accuracy of a classification network in low light and other challenging conditions, outperforming alternative approaches such as retraining the network on noisy and blurry images and preprocessing raw sensor inputs with conventional denoising and deblurring algorithms. The architecture learns denoising and deblurring pipelines optimized for classification whose outputs differ markedly from those of state-of-the-art denoising and deblurring methods, preserving fine detail at the cost of more noise and artifacts. Our results suggest that the best low-level image processing for computer vision is different from existing algorithms designed to produce visually pleasing images. The principles used to design the proposed architecture easily extend to other high-level computer vision tasks and image formation models, providing a general framework for integrating low-level and high-level image processing.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 9

research
06/14/2017

When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach

Conventionally, image denoising and high-level vision tasks are handled ...
research
09/09/2016

An empirical study on the effects of different types of noise in image classification tasks

Image classification is one of the main research problems in computer vi...
research
05/05/2023

Contrastive Learning for Low-light Raw Denoising

Image/video denoising in low-light scenes is an extremely challenging pr...
research
12/11/2014

High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial

In many applications, such as development and testing of image processin...
research
06/18/2021

Residual Contrastive Learning for Joint Demosaicking and Denoising

The breakthrough of contrastive learning (CL) has fueled the recent succ...
research
04/13/2018

FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification

Images captured by fisheye lenses violate the pinhole camera assumption ...
research
12/03/2019

Deep Learning based Switching Filter for Impulsive Noise Removal in Color Images

Noise reduction is one the most important and still active research topi...

Please sign up or login with your details

Forgot password? Click here to reset