Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention

by   Haiwei Wu, et al.

Deep learning (DL) has demonstrated its powerful capabilities in the field of image inpainting. The DL-based image inpainting approaches can produce visually plausible results, but often generate various unpleasant artifacts, especially in the boundary and highly textured regions. To tackle this challenge, in this work, we propose a new end-to-end, two-stage (coarse-to-fine) generative model through combining a local binary pattern (LBP) learning network with an actual inpainting network. Specifically, the first LBP learning network using U-Net architecture is designed to accurately predict the structural information of the missing region, which subsequently guides the second image inpainting network for better filling the missing pixels. Furthermore, an improved spatial attention mechanism is integrated in the image inpainting network, by considering the consistency not only between the known region with the generated one, but also within the generated region itself. Extensive experiments on public datasets including CelebA-HQ, Places and Paris StreetView demonstrate that our model generates better inpainting results than the state-of-the-art competing algorithms, both quantitatively and qualitatively. The source code and trained models will be made available at



There are no comments yet.


page 1

page 2

page 3

page 4

page 5

page 7

page 9

page 10


Deep Two-Stage High-Resolution Image Inpainting

In recent years, the field of image inpainting has developed rapidly, le...

GIID-Net: Generalizable Image Inpainting Detection via Neural Architecture Search and Attention

Deep learning (DL) has demonstrated its powerful capabilities in the fie...

DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting

The degree of difficulty in image inpainting depends on the types and si...

Diversity-Generated Image Inpainting with Style Extraction

The latest methods based on deep learning have achieved amazing results ...

Contextual Attention Mechanism, SRGAN Based Inpainting System for Eliminating Interruptions from Images

The new alternative is to use deep learning to inpaint any image by util...

Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness

Inpainting arbitrary missing regions is challenging because learning val...

Pixel-wise Dense Detector for Image Inpainting

Recent GAN-based image inpainting approaches adopt an average strategy t...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.