Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention

09/02/2020
by   Haiwei Wu, et al.
3

Deep learning (DL) has demonstrated its powerful capabilities in the field of image inpainting. The DL-based image inpainting approaches can produce visually plausible results, but often generate various unpleasant artifacts, especially in the boundary and highly textured regions. To tackle this challenge, in this work, we propose a new end-to-end, two-stage (coarse-to-fine) generative model through combining a local binary pattern (LBP) learning network with an actual inpainting network. Specifically, the first LBP learning network using U-Net architecture is designed to accurately predict the structural information of the missing region, which subsequently guides the second image inpainting network for better filling the missing pixels. Furthermore, an improved spatial attention mechanism is integrated in the image inpainting network, by considering the consistency not only between the known region with the generated one, but also within the generated region itself. Extensive experiments on public datasets including CelebA-HQ, Places and Paris StreetView demonstrate that our model generates better inpainting results than the state-of-the-art competing algorithms, both quantitatively and qualitatively. The source code and trained models will be made available at https://github.com/HighwayWu/ImageInpainting.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

page 5

page 7

page 9

page 10

04/27/2021

Deep Two-Stage High-Resolution Image Inpainting

In recent years, the field of image inpainting has developed rapidly, le...
01/19/2021

GIID-Net: Generalizable Image Inpainting Detection via Neural Architecture Search and Attention

Deep learning (DL) has demonstrated its powerful capabilities in the fie...
08/17/2020

DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting

The degree of difficulty in image inpainting depends on the types and si...
12/04/2019

Diversity-Generated Image Inpainting with Style Extraction

The latest methods based on deep learning have achieved amazing results ...
04/06/2022

Contextual Attention Mechanism, SRGAN Based Inpainting System for Eliminating Interruptions from Images

The new alternative is to use deep learning to inpaint any image by util...
04/28/2021

Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness

Inpainting arbitrary missing regions is challenging because learning val...
11/04/2020

Pixel-wise Dense Detector for Image Inpainting

Recent GAN-based image inpainting approaches adopt an average strategy t...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.