DeepAI AI Chat
Log In Sign Up

Learning Multi-modal Information for Robust Light Field Depth Estimation

by   Yongri Piao, et al.

Light field data has been demonstrated to facilitate the depth estimation task. Most learning-based methods estimate the depth infor-mation from EPI or sub-aperture images, while less methods pay attention to the focal stack. Existing learning-based depth estimation methods from the focal stack lead to suboptimal performance because of the defocus blur. In this paper, we propose a multi-modal learning method for robust light field depth estimation. We first excavate the internal spatial correlation by designing a context reasoning unit which separately extracts comprehensive contextual information from the focal stack and RGB images. Then we integrate the contextual information by exploiting a attention-guide cross-modal fusion module. Extensive experiments demonstrate that our method achieves superior performance than existing representative methods on two light field datasets. Moreover, visual results on a mobile phone dataset show that our method can be widely used in daily life.


page 4

page 6

page 9

page 11

page 13

page 14


Dynamic Fusion Network For Light Field Depth Estimation

Focus based methods have shown promising results for the task of depth e...

Depth estimation from 4D light field videos

Depth (disparity) estimation from 4D Light Field (LF) images has been a ...

View-consistent 4D Light Field Depth Estimation

We propose a method to compute depth maps for every sub-aperture image i...

CroMo: Cross-Modal Learning for Monocular Depth Estimation

Learning-based depth estimation has witnessed recent progress in multipl...

Beyond Image to Depth: Improving Depth Prediction using Echoes

We address the problem of estimating depth with multi modal audio visual...

EPI-based Oriented Relation Networks for Light Field Depth Estimation

Light fields record not only the spatial information of observed scenes ...

Deep Anti-aliasing of Whole Focal Stack Using its Slice Spectrum

The paper aims at removing the aliasing effects for the whole focal stac...