Learning Single Camera Depth Estimation using Dual-Pixels

04/11/2019
by   Rahul Garg, et al.
0

Deep learning techniques have enabled rapid progress in monocular depth estimation, but their quality is limited by the ill-posed nature of the problem and the scarcity of high quality datasets. We estimate depth from a single camera by leveraging the dual-pixel auto-focus hardware that is increasingly common on modern camera sensors. Classic stereo algorithms and prior learning-based depth estimation techniques under-perform when applied on this dual-pixel data, the former due to too-strong assumptions about RGB image matching, and the latter due to a lack of understanding of the optics of dual-pixel image formation. To allow learning based methods to work well on dual-pixel imagery, we identify an inherent ambiguity in the depth estimated from dual-pixel cues, and develop an approach to estimate depth up to this ambiguity. Using our approach, existing monocular depth estimation techniques can be effectively applied to dual-pixel data, and much smaller models can be constructed that still infer high quality depth. To demonstrate this, we capture a large dataset of in-the-wild 5-viewpoint RGB images paired with corresponding dual-pixel data, and show how view supervision with this data can be used to learn depth up to the unknown ambiguities. On our new task, our model is 30 stereoscopic depth estimation.

READ FULL TEXT

page 5

page 8

page 11

page 14

page 15

page 16

page 17

page 18

research
03/14/2022

Unsupervised Learning Based Focal Stack Camera Depth Estimation

We propose an unsupervised deep learning based method to estimate depth ...
research
03/31/2020

Du^2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels

Computational stereo has reached a high level of accuracy, but degrades ...
research
10/15/2018

Playing for Depth

Estimating the relative depth of a scene is a significant step towards u...
research
04/26/2020

Learning to Autofocus

Autofocus is an important task for digital cameras, yet current approach...
research
09/20/2021

Augmenting Depth Estimation with Geospatial Context

Modern cameras are equipped with a wide array of sensors that enable rec...
research
04/14/2022

End-to-end Learning for Joint Depth and Image Reconstruction from Diffracted Rotation

Monocular depth estimation is still an open challenge due to the ill-pos...
research
03/21/2018

Robust Depth Estimation from Auto Bracketed Images

As demand for advanced photographic applications on hand-held devices gr...

Please sign up or login with your details

Forgot password? Click here to reset