DeepAI
Log In Sign Up

Image Inpainting with Edge-guided Learnable Bidirectional Attention Maps

04/25/2021
by   Dongsheng Wang, et al.
0

For image inpainting, the convolutional neural networks (CNN) in previous methods often adopt standard convolutional operator, which treats valid pixels and holes indistinguishably. As a result, they are limited in handling irregular holes and tend to produce color-discrepant and blurry inpainting result. Partial convolution (PConv) copes with this issue by conducting masked convolution and feature re-normalization conditioned only on valid pixels, but the mask-updating is handcrafted and independent with image structural information. In this paper, we present an edge-guided learnable bidirectional attention map (Edge-LBAM) for improving image inpainting of irregular holes with several distinct merits. Instead of using a hard 0-1 mask, a learnable attention map module is introduced for learning feature re-normalization and mask-updating in an end-to-end manner. Learnable reverse attention maps are further proposed in the decoder for emphasizing on filling in unknown pixels instead of reconstructing all pixels. Motivated by that the filling-in order is crucial to inpainting results and largely depends on image structures in exemplar-based methods, we further suggest a multi-scale edge completion network to predict coherent edges. Our Edge-LBAM method contains dual procedures,including structure-aware mask-updating guided by predict edges and attention maps generated by masks for feature re-normalization.Extensive experiments show that our Edge-LBAM is effective in generating coherent image structures and preventing color discrepancy and blurriness, and performs favorably against the state-of-the-art methods in terms of qualitative metrics and visual quality.

READ FULL TEXT

page 2

page 4

page 7

page 10

page 11

page 12

page 13

page 14

09/03/2019

Image Inpainting with Learnable Bidirectional Attention Maps

Most convolutional network (CNN)-based inpainting methods adopt standard...
04/20/2018

Image Inpainting for Irregular Holes Using Partial Convolutions

Existing deep learning based image inpainting methods use a standard con...
09/10/2021

View Blind-spot as Inpainting: Self-Supervised Denoising with Mask Guided Residual Convolution

In recent years, self-supervised denoising methods have shown impressive...
04/28/2021

Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness

Inpainting arbitrary missing regions is challenging because learning val...
06/07/2022

Wavelet Prior Attention Learning in Axial Inpainting Network

Image inpainting is the task of filling masked or unknown regions of an ...
11/02/2020

Image Inpainting with Learnable Feature Imputation

A regular convolution layer applying a filter in the same way over known...
04/18/2022

Cylin-Painting: Seamless 360° Panoramic Image Outpainting and Beyond with Cylinder-Style Convolutions

Image outpainting gains increasing attention since it can generate the c...