Wavelet Prior Attention Learning in Axial Inpainting Network

06/07/2022
by   Chenjie Cao, et al.
0

Image inpainting is the task of filling masked or unknown regions of an image with visually realistic contents, which has been remarkably improved by Deep Neural Networks (DNNs) recently. Essentially, as an inverse problem, the inpainting has the underlying challenges of reconstructing semantically coherent results without texture artifacts. Many previous efforts have been made via exploiting attention mechanisms and prior knowledge, such as edges and semantic segmentation. However, these works are still limited in practice by an avalanche of learnable prior parameters and prohibitive computational burden. To this end, we propose a novel model – Wavelet prior attention learning in Axial Inpainting Network (WAIN), whose generator contains the encoder, decoder, as well as two key components of Wavelet image Prior Attention (WPA) and stacked multi-layer Axial-Transformers (ATs). Particularly, the WPA guides the high-level feature aggregation in the multi-scale frequency domain, alleviating the textual artifacts. Stacked ATs employ unmasked clues to help model reasonable features along with low-level features of horizontal and vertical axes, improving the semantic coherence. Extensive quantitative and qualitative experiments on Celeba-HQ and Places2 datasets are conducted to validate that our WAIN can achieve state-of-the-art performance over the competitors. The codes and models will be released.

READ FULL TEXT

page 1

page 7

page 8

page 11

page 12

page 13

research
04/16/2019

Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting

High-quality image inpainting requires filling missing regions in a dama...
research
07/15/2023

Both Spatial and Frequency Cues Contribute to High-Fidelity Image Inpainting

Deep generative approaches have obtained great success in image inpainti...
research
08/03/2022

Learning Prior Feature and Attention Enhanced Image Inpainting

Many recent inpainting works have achieved impressive results by leverag...
research
04/25/2021

Image Inpainting with Edge-guided Learnable Bidirectional Attention Maps

For image inpainting, the convolutional neural networks (CNN) in previou...
research
04/26/2021

Diverse Image Inpainting with Bidirectional and Autoregressive Transformers

Image inpainting is an underdetermined inverse problem, it naturally all...
research
12/08/2021

Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid

Restoring reasonable and realistic content for arbitrary missing regions...
research
07/08/2020

SiENet: Siamese Expansion Network for Image Extrapolation

Different from image inpainting, image outpainting has relative less con...

Please sign up or login with your details

Forgot password? Click here to reset