Deep Optics for Monocular Depth Estimation and 3D Object Detection

04/18/2019
by   Julie Chang, et al.
18

Depth estimation and 3D object detection are critical for scene understanding but remain challenging to perform with a single image due to the loss of 3D information during image capture. Recent models using deep neural networks have improved monocular depth estimation performance, but there is still difficulty in predicting absolute depth and generalizing outside a standard dataset. Here we introduce the paradigm of deep optics, i.e. end-to-end design of optics and image processing, to the monocular depth estimation problem, using coded defocus blur as an additional depth cue to be decoded by a neural network. We evaluate several optical coding strategies along with an end-to-end optimization scheme for depth estimation on three datasets, including NYU Depth v2 and KITTI. We find an optimized freeform lens design yields the best results, but chromatic aberration from a singlet lens offers significantly improved performance as well. We build a physical prototype and validate that chromatic aberrations improve depth estimation on real-world results. In addition, we train object detection networks on the KITTI dataset and show that the lens optimized for depth estimation also results in improved 3D object detection performance.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

research
11/02/2021

Absolute distance prediction based on deep learning object detection and monocular depth estimation models

Determining the distance between the objects in a scene and the camera s...
research
02/17/2023

Long Range Object-Level Monocular Depth Estimation for UAVs

Computer vision-based object detection is a key modality for advanced De...
research
04/24/2022

RealNet: Combining Optimized Object Detection with Information Fusion Depth Estimation Co-Design Method on IoT

Depth Estimation and Object Detection Recognition play an important role...
research
09/14/2023

Depth Estimation from a Single Optical Encoded Image using a Learned Colored-Coded Aperture

Depth estimation from a single image of a conventional camera is a chall...
research
02/20/2023

On the Metrics for Evaluating Monocular Depth Estimation

Monocular Depth Estimation (MDE) is performed to produce 3D information ...
research
08/25/2021

Monocular Depth Estimation Primed by Salient Point Detection and Normalized Hessian Loss

Deep neural networks have recently thrived on single image depth estimat...
research
04/13/2022

Does depth estimation help object detection?

Ground-truth depth, when combined with color data, helps improve object ...

Please sign up or login with your details

Forgot password? Click here to reset