RGB-D Image Inpainting Using Generative Adversarial Network with a Late Fusion Approach

10/14/2021
by   Ryo Fujii, et al.
11

Diminished reality is a technology that aims to remove objects from video images and fills in the missing region with plausible pixels. Most conventional methods utilize the different cameras that capture the same scene from different viewpoints to allow regions to be removed and restored. In this paper, we propose an RGB-D image inpainting method using generative adversarial network, which does not require multiple cameras. Recently, an RGB image inpainting method has achieved outstanding results by employing a generative adversarial network. However, RGB inpainting methods aim to restore only the texture of the missing region and, therefore, does not recover geometric information (i.e, 3D structure of the scene). We expand conventional image inpainting method to RGB-D image inpainting to jointly restore the texture and geometry of missing regions from a pair of RGB and depth images. Inspired by other tasks that use RGB and depth images (e.g., semantic segmentation and object detection), we propose late fusion approach that exploits the advantage of RGB and depth information each other. The experimental results verify the effectiveness of our proposed method.

READ FULL TEXT

page 8

page 9

page 10

research
03/20/2018

Patch-Based Image Inpainting with Generative Adversarial Networks

Area of image inpainting over relatively large missing regions recently ...
research
05/29/2018

Semantic Road Layout Understanding by Generative Adversarial Inpainting

Autonomous driving is becoming a reality, yet vehicles still need to rel...
research
08/12/2020

Dynamic Object Removal and Spatio-Temporal RGB-D Inpainting via Geometry-Aware Adversarial Learning

Dynamic objects have a significant impact on the robot's perception of t...
research
04/16/2018

Defo-Net: Learning Body Deformation using Generative Adversarial Networks

Modelling the physical properties of everyday objects is a fundamental p...
research
11/22/2018

Multi-View Inpainting for RGB-D Sequence

In this work we propose a novel approach to remove undesired objects fro...
research
08/29/2021

Flow-Guided Video Inpainting with Scene Templates

We consider the problem of filling in missing spatio-temporal regions of...
research
03/28/2019

GANs-NQM: A Generative Adversarial Networks based No Reference Quality Assessment Metric for RGB-D Synthesized Views

In this paper, we proposed a no-reference (NR) quality metric for RGB pl...

Please sign up or login with your details

Forgot password? Click here to reset