TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations

by   Yuqian Zhou, et al.

Image inpainting is the task of plausibly restoring missing pixels within a hole region that is to be removed from a target image. Most existing technologies exploit patch similarities within the image, or leverage large-scale training data to fill the hole using learned semantic and texture information. However, due to the ill-posed nature of the inpainting task, such methods struggle to complete larger holes containing complicated scenes. In this paper, we propose TransFill, a multi-homography transformed fusion method to fill the hole by referring to another source image that shares scene contents with the target image. We first align the source image to the target image by estimating multiple homographies guided by different depth levels. We then learn to adjust the color and apply a pixel-level warping to each homography-warped source image to make it more consistent with the target. Finally, a pixel-level fusion module is learned to selectively merge the different proposals. Our method achieves state-of-the-art performance on pairs of images across a variety of wide baselines and color differences, and generalizes to user-provided image pairs.


page 1

page 7

page 12

page 13

page 15

page 16

page 17

page 18


Reference-Guided Texture and Structure Inference for Image Inpainting

Existing learning-based image inpainting methods are still in challenge ...

GeoFill: Reference-Based Image Inpainting of Scenes with Complex Geometry

Reference-guided image inpainting restores image pixels by leveraging th...

Large Hole Image Inpainting With Compress-Decompression Network

Image inpainting technology can patch images with missing pixels. Existi...

Non-Local Robust Quaternion Matrix Completion for Large-Scale Color Images and Videos Inpainting

The image nonlocal self-similarity (NSS) prior refers to the fact that a...

Instance-aware Image Colorization

Image colorization is inherently an ill-posed problem with multi-modal u...

Offset equivariant networks and their applications

In this paper we present a framework for the design and implementation o...

Onion-Peel Networks for Deep Video Completion

We propose the onion-peel networks for video completion. Given a set of ...