Hardware based Spatio-Temporal Neural Processing Backend for Imaging Sensors: Towards a Smart Camera

03/23/2018
by   Samiran Ganguly, et al.
0

In this work we show how we can build a technology platform for cognitive imaging sensors using recent advances in recurrent neural network architectures and training methods inspired from biology. We demonstrate learning and processing tasks specific to imaging sensors, including enhancement of sensitivity and signal-to-noise ratio (SNR) purely through neural filtering beyond the fundamental limits sensor materials, and inferencing and spatio-temporal pattern recognition capabilities of these networks with applications in object detection, motion tracking and prediction. We then show designs of unit hardware cells built using complementary metal-oxide semiconductor (CMOS) and emerging materials technologies for ultra-compact and energy-efficient embedded neural processors for smart cameras.

READ FULL TEXT

page 2

page 6

research
04/11/2023

PixelRNN: In-pixel Recurrent Neural Networks for End-to-end-optimized Perception with Neural Sensors

Conventional image sensors digitize high-resolution images at fast frame...
research
05/30/2022

Braille Letter Reading: A Benchmark for Spatio-Temporal Pattern Recognition on Neuromorphic Hardware

Spatio-temporal pattern recognition is a fundamental ability of the brai...
research
03/14/2022

GradTac: Spatio-Temporal Gradient Based Tactile Sensing

Tactile sensing for robotics is achieved through a variety of mechanisms...
research
09/29/2017

Reservoir Computing using Stochastic p-Bits

We present a general hardware framework for building networks that direc...
research
07/06/2020

Building Reservoir Computing Hardware Using Low Energy-Barrier Magnetics

Biologically inspired recurrent neural networks, such as reservoir compu...
research
11/06/2018

A `Little Bit' Too Much? High Speed Imaging from Sparse Photon Counts

Recent advances in photographic sensing technologies have made it possib...

Please sign up or login with your details

Forgot password? Click here to reset