Physics vs. Learned Priors: Rethinking Camera and Algorithm Design for Task-Specific Imaging

04/21/2022
by   Tzofi Klinghoffer, et al.
0

Cameras were originally designed using physics-based heuristics to capture aesthetic images. In recent years, there has been a transformation in camera design from being purely physics-driven to increasingly data-driven and task-specific. In this paper, we present a framework to understand the building blocks of this nascent field of end-to-end design of camera hardware and algorithms. As part of this framework, we show how methods that exploit both physics and data have become prevalent in imaging and computer vision, underscoring a key trend that will continue to dominate the future of task-specific camera design. Finally, we share current barriers to progress in end-to-end design, and hypothesize how these barriers can be overcome.

READ FULL TEXT

page 2

page 4

research
04/28/2023

Differentiable Sensor Layouts for End-to-End Learning of Task-Specific Camera Parameters

The success of deep learning is frequently described as the ability to t...
research
05/26/2022

To image, or not to image: Class-specific diffractive cameras with all-optical erasure of undesired objects

Privacy protection is a growing concern in the digital era, with machine...
research
03/29/2022

Synergizing Physics/Model-based and Data-driven Methods for Low-Dose CT

Since 2016, deep learning (DL) has advanced tomographic imaging with rem...
research
11/15/2022

Autonomous Golf Putting with Data-Driven and Physics-Based Methods

We are developing a self-learning mechatronic golf robot using combined ...
research
09/28/2016

Learning to Push by Grasping: Using multiple tasks for effective learning

Recently, end-to-end learning frameworks are gaining prevalence in the f...
research
03/04/2020

Redesigning SLAM for Arbitrary Multi-Camera Systems

Adding more cameras to SLAM systems improves robustness and accuracy but...
research
09/24/2018

Vision-based Control of a Quadrotor in User Proximity: Mediated vs End-to-End Learning Approaches

We consider the task of controlling a quadrotor to hover in front of a f...

Please sign up or login with your details

Forgot password? Click here to reset