High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis

11/30/2016
by   Chao Yang, et al.
0

Recent advances in deep learning have shown exciting promise in filling large holes in natural images with semantically plausible and context aware details, impacting fundamental image manipulation tasks such as object removal. While these learning-based methods are significantly more effective in capturing high-level features than prior techniques, they can only handle very low-resolution inputs due to memory limitations and difficulty in training. Even for slightly larger images, the inpainted regions would appear blurry and unpleasant boundaries become visible. We propose a multi-scale neural patch synthesis approach based on joint optimization of image content and texture constraints, which not only preserves contextual structures but also produces high-frequency details by matching and adapting patches with the most similar mid-layer feature correlations of a deep classification network. We evaluate our method on the ImageNet and Paris Streetview datasets and achieved state-of-the-art inpainting accuracy. We show our approach produces sharper and more coherent results than prior methods, especially for high-resolution images.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

research
05/19/2020

Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting

Recently data-driven image inpainting methods have made inspiring progre...
research
12/08/2017

Image Inpainting for High-Resolution Textures using CNN Texture Synthesis

Deep neural networks have been successfully applied to problems such as ...
research
11/23/2017

Image Inpainting using Multi-Scale Feature Image Translation

We study the task of image inpainting, which is to fill in the missing r...
research
02/07/2022

Patch-Based Stochastic Attention for Image Editing

Attention mechanisms have become of crucial importance in deep learning ...
research
12/01/2018

Deep Inception Generative Network for Cognitive Image Inpainting

Recent advances in deep learning have shown exciting promise in filling ...
research
12/18/2019

One-Stage Inpainting with Bilateral Attention and Pyramid Filling Block

Recent deep learning based image inpainting methods which utilize contex...
research
10/29/2020

Free-Form Image Inpainting via Contrastive Attention Network

Most deep learning based image inpainting approaches adopt autoencoder o...

Please sign up or login with your details

Forgot password? Click here to reset