High-speed object detection with a single-photon time-of-flight image sensor

07/28/2021
by   Germán Mora-Martín, et al.
6

3D time-of-flight (ToF) imaging is used in a variety of applications such as augmented reality (AR), computer interfaces, robotics and autonomous systems. Single-photon avalanche diodes (SPADs) are one of the enabling technologies providing accurate depth data even over long ranges. By developing SPADs in array format with integrated processing combined with pulsed, flood-type illumination, high-speed 3D capture is possible. However, array sizes tend to be relatively small, limiting the lateral resolution of the resulting depth maps, and, consequently, the information that can be extracted from the image for applications such as object detection. In this paper, we demonstrate that these limitations can be overcome through the use of convolutional neural networks (CNNs) for high-performance object detection. We present outdoor results from a portable SPAD camera system that outputs 16-bin photon timing histograms with 64x32 spatial resolution. The results, obtained with exposure times down to 2 ms (equivalent to 500 FPS) and in signal-to-background (SBR) ratios as low as 0.05, point to the advantages of providing the CNN with full histogram data rather than point clouds alone. Alternatively, a combination of point cloud and active intensity data may be used as input, for a similar level of performance. In either case, the GPU-accelerated processing time is less than 1 ms per frame, leading to an overall latency (image acquisition plus processing) in the millisecond range, making the results relevant for safety-critical computer vision applications which would benefit from faster than human reaction times.

READ FULL TEXT

page 4

page 5

page 6

page 10

research
10/19/2022

Video super-resolution for single-photon LIDAR

3D Time-of-Flight (ToF) image sensors are used widely in applications su...
research
11/19/2018

FotonNet: A HW-Efficient Object Detection System Using 3D-Depth Segmentation and 2D-DNN Classifier

Object detection and classification is one of the most important compute...
research
09/23/2022

A direct time-of-flight image sensor with in-pixel surface detection and dynamic vision

3D flash LIDAR is an alternative to the traditional scanning LIDAR syste...
research
09/30/2022

PointPillars Backbone Type Selection For Fast and Accurate LiDAR Object Detection

3D object detection from LiDAR sensor data is an important topic in the ...
research
05/04/2020

Streaming Object Detection for 3-D Point Clouds

Autonomous vehicles operate in a dynamic environment, where the speed wi...
research
10/21/2021

A Fast Location Algorithm for Very Sparse Point Clouds Based on Object Detection

Limited by the performance factor, it is arduous to recognize target obj...
research
07/22/2015

Bayesian Time-of-Flight for Realtime Shape, Illumination and Albedo

We propose a computational model for shape, illumination and albedo infe...

Please sign up or login with your details

Forgot password? Click here to reset