Bio-inspired Polarization Event Camera

by   Germain Haessig, et al.

The stomatopod (mantis shrimp) visual system has recently provided a blueprint for the design of paradigm-shifting polarization and multispectral imaging sensors, enabling solutions to challenging medical and remote sensing problems. However, these bioinspired sensors lack the high dynamic range (HDR) and asynchronous polarization vision capabilities of the stomatopod visual system, limiting temporal resolution to 1̃2 ms and dynamic range to 7̃2 dB. Here we present a novel stomatopod-inspired polarization camera which mimics the sustained and transient biological visual pathways to save power and sample data beyond the maximum Nyquist frame rate. This bio-inspired sensor simultaneously captures both synchronous intensity frames and asynchronous polarization brightness change information with sub-millisecond latencies over a million-fold range of illumination. Our PDAVIS camera is comprised of 346x260 pixels, organized in 2-by-2 macropixels, which filter the incoming light with four linear polarization filters offset by 45 degrees. Polarization information is reconstructed using both low cost and latency event-based algorithms and more accurate but slower deep neural networks. Our sensor is used to image HDR polarization scenes which vary at high speeds and to observe dynamical properties of single collagen fibers in bovine tendon under rapid cyclical loads



There are no comments yet.


page 2

page 3

page 4

page 5

page 6

page 11

page 13

page 16


A Linear Comb Filter for Event Flicker Removal

Event cameras are bio-inspired sensors that capture per-pixel asynchrono...

An Asynchronous Kalman Filter for Hybrid Event Cameras

We present an Asynchronous Kalman Filter (AKF) to reconstruct High Dynam...

The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM

New vision sensors, such as the Dynamic and Active-pixel Vision sensor (...

Design and Implementation of a Novel Compatible Encoding Scheme in the Time Domain for Image Sensor Communication

This paper presents a modulation scheme in the time domain based on On-O...

Learning to Super Resolve Intensity Images from Events

An event camera detects per-pixel intensity difference and produces asyn...

Passive Inter-Photon Imaging

Digital camera pixels measure image intensities by converting incident l...

Using Spatially Varying Pixels Exposures and Bayer-covered Photosensors for High Dynamic Range Imaging

The method of a linear high dynamic range imaging using solid-state phot...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Abstract

The stomatopod (mantis shrimp) visual system  Marshall88,cronin2014visual,Horvath2014-polarization-light-and-vision-in-animals has recently provided a blueprint for the design of paradigm-shifting polarization and multispectral imaging sensors  Altaquieabe3196,GarciaPolarization,BioinspiredSenorsReview2021,jen2011biologically,LiuEndoscope,Blaireaaw7067, ZhangPolarization, enabling solutions to challenging medical LiuEndoscope,Blaireaaw7067 and remote sensing problems Powelleaao6841. However, these bioinspired sensors lack the