DeepAI AI Chat
Log In Sign Up

I See-Through You: A Framework for Removing Foreground Occlusion in Both Sparse and Dense Light Field Images

by   Jiwan Hur, et al.
KAIST 수리과학과

Light field (LF) camera captures rich information from a scene. Using the information, the LF de-occlusion (LF-DeOcc) task aims to reconstruct the occlusion-free center view image. Existing LF-DeOcc studies mainly focus on the sparsely sampled (sparse) LF images where most of the occluded regions are visible in other views due to the large disparity. In this paper, we expand LF-DeOcc in more challenging datasets, densely sampled (dense) LF images, which are taken by a micro-lens-based portable LF camera. Due to the small disparity ranges of dense LF images, most of the background regions are invisible in any view. To apply LF-DeOcc in both LF datasets, we propose a framework, ISTY, which is defined and divided into three roles: (1) extract LF features, (2) define the occlusion, and (3) inpaint occluded regions. By dividing the framework into three specialized components according to the roles, the development and analysis can be easier. Furthermore, an explainable intermediate representation, an occlusion mask, can be obtained in the proposed framework. The occlusion mask is useful for comprehensive analysis of the model and other applications by manipulating the mask. In experiments, qualitative and quantitative results show that the proposed framework outperforms state-of-the-art LF-DeOcc methods in both sparse and dense LF datasets.


page 14

page 15

page 16

page 17

page 18

page 19

page 21

page 22


DeOccNet: Learning to See Through Foreground Occlusions in Light Fields

Background objects occluded in some views of a light field (LF) camera c...

Unsupervised Light Field Depth Estimation via Multi-view Feature Matching with Occlusion Prediction

Depth estimation from light field (LF) images is a fundamental step for ...

Learning Occlusion-Aware View Synthesis for Light Fields

In this work, we present a novel learning-based approach to synthesize n...

Variational Disparity Estimation Framework for Plenoptic Image

This paper presents a computational framework for accurately estimating ...

Stratified Labeling for Surface Consistent Parallax Correction and Occlusion Completion

The light field faithfully records the spatial and angular configuration...

Geometry-based Occlusion-Aware Unsupervised Stereo Matching for Autonomous Driving

Recently, there are emerging many stereo matching methods for autonomous...

Stereoscopic Neural Style Transfer

This paper presents the first attempt at stereoscopic neural style trans...