Learning Single Camera Depth Estimation using Dual-Pixels

by   Rahul Garg, et al.

Deep learning techniques have enabled rapid progress in monocular depth estimation, but their quality is limited by the ill-posed nature of the problem and the scarcity of high quality datasets. We estimate depth from a single camera by leveraging the dual-pixel auto-focus hardware that is increasingly common on modern camera sensors. Classic stereo algorithms and prior learning-based depth estimation techniques under-perform when applied on this dual-pixel data, the former due to too-strong assumptions about RGB image matching, and the latter due to a lack of understanding of the optics of dual-pixel image formation. To allow learning based methods to work well on dual-pixel imagery, we identify an inherent ambiguity in the depth estimated from dual-pixel cues, and develop an approach to estimate depth up to this ambiguity. Using our approach, existing monocular depth estimation techniques can be effectively applied to dual-pixel data, and much smaller models can be constructed that still infer high quality depth. To demonstrate this, we capture a large dataset of in-the-wild 5-viewpoint RGB images paired with corresponding dual-pixel data, and show how view supervision with this data can be used to learn depth up to the unknown ambiguities. On our new task, our model is 30 stereoscopic depth estimation.


page 5

page 8

page 11

page 14

page 15

page 16

page 17

page 18


Unsupervised Learning Based Focal Stack Camera Depth Estimation

We propose an unsupervised deep learning based method to estimate depth ...

Du^2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels

Computational stereo has reached a high level of accuracy, but degrades ...

Playing for Depth

Estimating the relative depth of a scene is a significant step towards u...

Learning to Autofocus

Autofocus is an important task for digital cameras, yet current approach...

Augmenting Depth Estimation with Geospatial Context

Modern cameras are equipped with a wide array of sensors that enable rec...

End-to-end Learning for Joint Depth and Image Reconstruction from Diffracted Rotation

Monocular depth estimation is still an open challenge due to the ill-pos...

Robust Depth Estimation from Auto Bracketed Images

As demand for advanced photographic applications on hand-held devices gr...

Please sign up or login with your details

Forgot password? Click here to reset