DeepAI AI Chat
Log In Sign Up

Naturalizing Neuromorphic Vision Event Streams Using GANs

by   Dennis Robey, et al.

Dynamic vision sensors are able to operate at high temporal resolutions within resource constrained environments, though at the expense of capturing static content. The sparse nature of event streams enables efficient downstream processing tasks as they are suited for power-efficient spiking neural networks. One of the challenges associated with neuromorphic vision is the lack of interpretability of event streams. While most application use-cases do not intend for the event stream to be visually interpreted by anything other than a classification network, there is a lost opportunity to integrating these sensors in spaces that conventional high-speed CMOS sensors cannot go. For example, biologically invasive sensors such as endoscopes must fit within stringent power budgets, which do not allow MHz-speeds of image integration. While dynamic vision sensing can fill this void, the interpretation challenge remains and will degrade confidence in clinical diagnostics. The use of generative adversarial networks presents a possible solution to overcoming and compensating for a vision chip's poor spatial resolution and lack of interpretability. In this paper, we methodically apply the Pix2Pix network to naturalize the event stream from spike-converted CIFAR-10 and Linnaeus 5 datasets. The quality of the network is benchmarked by performing image classification of naturalized event streams, which converges to within 2.81 equivalent raw images, and an associated improvement over unprocessed event streams by 13.19


page 1

page 2

page 4


ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks

With event-driven algorithms, especially the spiking neural networks (SN...

Event-driven Vision and Control for UAVs on a Neuromorphic Chip

Event-based vision sensors achieve up to three orders of magnitude bette...

EventMix: An Efficient Augmentation Strategy for Event-Based Data

High-quality and challenging event stream datasets play an important rol...

RecNet: Early Attention Guided Feature Recovery

Uncertainty in sensors results in corrupted input streams and hinders th...

CERiL: Continuous Event-based Reinforcement Learning

This paper explores the potential of event cameras to enable continuous ...

Person Re-Identification without Identification via Event Anonymization

Wide-scale use of visual surveillance in public spaces puts individual p...