Region Normalization for Image Inpainting

11/23/2019
by   Tao Yu, et al.
17

Feature Normalization (FN) is an important technique to help neural network training, which typically normalizes features across spatial dimensions. Most previous image inpainting methods apply FN in their networks without considering the impact of the corrupted regions of the input image on normalization, e.g. mean and variance shifts. In this work, we show that the mean and variance shifts caused by full-spatial FN limit the image inpainting network training and we propose a spatial region-wise normalization named Region Normalization (RN) to overcome the limitation. RN divides spatial pixels into different regions according to the input mask, and computes the mean and variance in each region for normalization. We develop two kinds of RN for our image inpainting network: (1) Basic RN (RN-B), which normalizes pixels from the corrupted and uncorrupted regions separately based on the original inpainting mask to solve the mean and variance shift problem; (2) Learnable RN (RN-L), which automatically detects potentially corrupted and uncorrupted regions for separate normalization, and performs global affine transformation to enhance their fusion. We apply RN-B in the early layers and RN-L in the latter layers of the network respectively. Experiments show that our method outperforms current state-of-the-art methods quantitatively and qualitatively. We further generalize RN to other inpainting networks and achieve consistent performance improvements.

READ FULL TEXT

page 5

page 6

page 7

page 8

research
06/01/2013

Image Inpainting by Kriging Interpolation Technique

Image inpainting is the art of predicting damaged regions of an image. T...
research
08/13/2018

Image Inpainting Based on a Novel Criminisi Algorithm

In view of the problem of image inpainting error continuation and the de...
research
12/25/2017

Deep Blind Image Inpainting

Image inpainting is a challenging problem as it needs to fill the inform...
research
04/28/2021

Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness

Inpainting arbitrary missing regions is challenging because learning val...
research
04/25/2021

Image Inpainting with Edge-guided Learnable Bidirectional Attention Maps

For image inpainting, the convolutional neural networks (CNN) in previou...
research
04/06/2022

ShowFace: Coordinated Face Inpainting with Memory-Disentangled Refinement Networks

Face inpainting aims to complete the corrupted regions of the face image...
research
11/02/2020

Image Inpainting with Learnable Feature Imputation

A regular convolution layer applying a filter in the same way over known...

Please sign up or login with your details

Forgot password? Click here to reset