Spiking Optical Flow for Event-based Sensors Using IBM's TrueNorth Neurosynaptic System

10/26/2017
by   Germain Haessig, et al.
0

This paper describes a fully spike-based neural network for optical flow estimation from Dynamic Vision Sensor data. A low power embedded implementation of the method which combines the Asynchronous Time-based Image Sensor with IBM's TrueNorth Neurosynaptic System is presented. The sensor generates spikes with sub-millisecond resolution in response to scene illumination changes. These spike are processed by a spiking neural network running on TrueNorth with a 1 millisecond resolution to accurately determine the order and time difference of spikes from neighboring pixels, and therefore infer the velocity. The spiking neural network is a variant of the Barlow Levick method for optical flow estimation. The system is evaluated on two recordings for which ground truth motion is available, and achieves an Average Endpoint Error of 11 estimated power budget of under 80mW for the sensor and computation.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 6

page 8

page 10

page 11

research
02/13/2023

Optical Flow estimation with Event-based Cameras and Spiking Neural Networks

Event-based cameras are raising interest within the computer vision comm...
research
04/11/2020

Bio-inspired Gait Imitation of Hexapod Robot Using Event-Based Vision Sensor and Spiking Neural Network

Learning how to walk is a sophisticated neurological task for most anima...
research
01/22/2019

Ego-motion Sensor for Unmanned Aerial Vehicles Based on a Single-Board Computer

This paper describes the design and implementation of a ground-related o...
research
05/29/2021

Foveal-pit inspired filtering of DVS spike response

In this paper, we present results of processing Dynamic Vision Sensor (D...
research
10/03/2016

Seeing into Darkness: Scotopic Visual Recognition

Images are formed by counting how many photons traveling from a given se...
research
06/16/2017

Block-Matching Optical Flow for Dynamic Vision Sensor- Algorithm and FPGA Implementation

Rapid and low power computation of optical flow (OF) is potentially usef...

Please sign up or login with your details

Forgot password? Click here to reset