DeepAI AI Chat
Log In Sign Up

Learning Multi-modal Information for Robust Light Field Depth Estimation

04/13/2021
by   Yongri Piao, et al.
0

Light field data has been demonstrated to facilitate the depth estimation task. Most learning-based methods estimate the depth infor-mation from EPI or sub-aperture images, while less methods pay attention to the focal stack. Existing learning-based depth estimation methods from the focal stack lead to suboptimal performance because of the defocus blur. In this paper, we propose a multi-modal learning method for robust light field depth estimation. We first excavate the internal spatial correlation by designing a context reasoning unit which separately extracts comprehensive contextual information from the focal stack and RGB images. Then we integrate the contextual information by exploiting a attention-guide cross-modal fusion module. Extensive experiments demonstrate that our method achieves superior performance than existing representative methods on two light field datasets. Moreover, visual results on a mobile phone dataset show that our method can be widely used in daily life.

READ FULL TEXT

page 4

page 6

page 9

page 11

page 13

page 14

04/13/2021

Dynamic Fusion Network For Light Field Depth Estimation

Focus based methods have shown promising results for the task of depth e...
12/05/2020

Depth estimation from 4D light field videos

Depth (disparity) estimation from 4D Light Field (LF) images has been a ...
09/09/2020

View-consistent 4D Light Field Depth Estimation

We propose a method to compute depth maps for every sub-aperture image i...
03/23/2022

CroMo: Cross-Modal Learning for Monocular Depth Estimation

Learning-based depth estimation has witnessed recent progress in multipl...
03/15/2021

Beyond Image to Depth: Improving Depth Prediction using Echoes

We address the problem of estimating depth with multi modal audio visual...
07/09/2020

EPI-based Oriented Relation Networks for Light Field Depth Estimation

Light fields record not only the spatial information of observed scenes ...
01/23/2021

Deep Anti-aliasing of Whole Focal Stack Using its Slice Spectrum

The paper aims at removing the aliasing effects for the whole focal stac...