Free-Form Image Inpainting via Contrastive Attention Network

10/29/2020
by   Xin Ma, et al.
14

Most deep learning based image inpainting approaches adopt autoencoder or its variants to fill missing regions in images. Encoders are usually utilized to learn powerful representational spaces, which are important for dealing with sophisticated learning tasks. Specifically, in image inpainting tasks, masks with any shapes can appear anywhere in images (i.e., free-form masks) which form complex patterns. It is difficult for encoders to capture such powerful representations under this complex situation. To tackle this problem, we propose a self-supervised Siamese inference network to improve the robustness and generalization. It can encode contextual semantics from full resolution images and obtain more discriminative representations. we further propose a multi-scale decoder with a novel dual attention fusion module (DAF), which can combine both the restored and known regions in a smooth way. This multi-scale architecture is beneficial for decoding discriminative representations learned by encoders into images layer by layer. In this way, unknown regions will be filled naturally from outside to inside. Qualitative and quantitative experiments on multiple datasets, including facial and natural datasets (i.e., Celeb-HQ, Pairs Street View, Places2 and ImageNet), demonstrate that our proposed method outperforms state-of-the-art methods in generating high-quality inpainting results.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

12/20/2021

Contrastive Attention Network with Dense Field Estimation for Face Completion

Most modern face completion approaches adopt an autoencoder or its varia...
02/23/2022

Multi-scale Sparse Representation-Based Shadow Inpainting for Retinal OCT Images

Inpainting shadowed regions cast by superficial blood vessels in retinal...
08/17/2020

DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting

The degree of difficulty in image inpainting depends on the types and si...
01/17/2018

Light-weight pixel context encoders for image inpainting

In this work we propose Pixel Content Encoders (PCE), a light-weight ima...
06/10/2018

Free-Form Image Inpainting with Gated Convolution

We present a novel deep learning based image inpainting system to comple...
05/08/2022

Iterative Geometry-Aware Cross Guidance Network for Stereo Image Inpainting

Currently, single image inpainting has achieved promising results based ...
07/01/2019

Global Transformer U-Nets for Label-Free Prediction of Fluorescence Images

Visualizing the details of different cellular structures is of great imp...