Exploiting Spatial Sparsity for Event Cameras with Visual Transformers

02/10/2022
by   Zuowen Wang, et al.
0

Event cameras report local changes of brightness through an asynchronous stream of output events. Events are spatially sparse at pixel locations with little brightness variation. We propose using a visual transformer (ViT) architecture to leverage its ability to process a variable-length input. The input to the ViT consists of events that are accumulated into time bins and spatially separated into non-overlapping sub-regions called patches. Patches are selected when the number of nonzero pixel locations within a sub-region is above a threshold. We show that by fine-tuning a ViT model on the selected active patches, we can reduce the average number of patches fed into the backbone during the inference by at least 50 the classification accuracy on the N-Caltech101 dataset. This reduction translates into a decrease of 51 an increase of 46

READ FULL TEXT
research
04/11/2022

Event Transformer

The event camera is a bio-vision inspired camera with high dynamic range...
research
10/29/2020

Asynchronous Corner Tracking Algorithm based on Lifetime of Events for DAVIS Cameras

Event cameras, i.e., the Dynamic and Active-pixel Vision Sensor (DAVIS) ...
research
12/17/2020

Event Camera Calibration of Per-pixel Biased Contrast Threshold

Event cameras output asynchronous events to represent intensity changes ...
research
04/07/2020

Real-time Classification from Short Event-Camera Streams using Input-filtering Neural ODEs

Event-based cameras are novel, efficient sensors inspired by the human v...
research
11/02/2022

The Lottery Ticket Hypothesis for Vision Transformers

The conventional lottery ticket hypothesis (LTH) claims that there exist...
research
11/22/2022

Event Transformer+. A multi-purpose solution for efficient event data processing

Event cameras record sparse illumination changes with high temporal reso...
research
05/02/2021

Feedback control of event cameras

Dynamic vision sensor event cameras produce a variable data rate stream ...

Please sign up or login with your details

Forgot password? Click here to reset