Event-driven Vision and Control for UAVs on a Neuromorphic Chip

08/08/2021
by   Antonio Vitale, et al.
0

Event-based vision sensors achieve up to three orders of magnitude better speed vs. power consumption trade off in high-speed control of UAVs compared to conventional image sensors. Event-based cameras produce a sparse stream of events that can be processed more efficiently and with a lower latency than images, enabling ultra-fast vision-driven control. Here, we explore how an event-based vision algorithm can be implemented as a spiking neuronal network on a neuromorphic chip and used in a drone controller. We show how seamless integration of event-based perception on chip leads to even faster control rates and lower latency. In addition, we demonstrate how online adaptation of the SNN controller can be realised using on-chip learning. Our spiking neuronal network on chip is the first example of a neuromorphic vision-based controller solving a high-speed UAV control task. The excellent scalability of processing in neuromorphic hardware opens the possibility to solve more challenging visual tasks in the future and integrate visual perception in fast control loops.

READ FULL TEXT
research
08/11/2021

Towards neuromorphic control: A spiking neural network based PID controller for UAV

In this work, we present a spiking neural network (SNN) based PID contro...
research
08/11/2021

Event-based PID controller fully realized in neuromorphic hardware: a one DoF study

Spiking Neuronal Networks (SNNs) realized in neuromorphic hardware lead ...
research
02/14/2021

Naturalizing Neuromorphic Vision Event Streams Using GANs

Dynamic vision sensors are able to operate at high temporal resolutions ...
research
01/05/2022

Neuromorphic Vision Based Control for the Precise Positioning of Robotic Drilling Systems

The manufacturing industry is currently witnessing a paradigm shift with...
research
08/18/2022

Kraken: A Direct Event/Frame-Based Multi-sensor Fusion SoC for Ultra-Efficient Visual Processing in Nano-UAVs

Small-size unmanned aerial vehicles (UAV) have the potential to dramatic...
research
03/24/2023

Object Motion Sensitivity: A Bio-inspired Solution to the Ego-motion Problem for Event-based Cameras

Neuromorphic (event-based) image sensors draw inspiration from the human...
research
04/08/2023

High Speed Neuromorphic Vision-Based Inspection of Countersinks in Automated Manufacturing Processes

Countersink inspection is crucial in various automated assembly lines, e...

Please sign up or login with your details

Forgot password? Click here to reset