Context Encoders: Feature Learning by Inpainting

04/25/2016
by   Deepak Pathak, et al.
0

We present an unsupervised visual feature learning algorithm driven by context-based pixel prediction. By analogy with auto-encoders, we propose Context Encoders -- a convolutional neural network trained to generate the contents of an arbitrary image region conditioned on its surroundings. In order to succeed at this task, context encoders need to both understand the content of the entire image, as well as produce a plausible hypothesis for the missing part(s). When training context encoders, we have experimented with both a standard pixel-wise reconstruction loss, as well as a reconstruction plus an adversarial loss. The latter produces much sharper results because it can better handle multiple modes in the output. We found that a context encoder learns a representation that captures not just appearance but also the semantics of visual structures. We quantitatively demonstrate the effectiveness of our learned features for CNN pre-training on classification, detection, and segmentation tasks. Furthermore, context encoders can be used for semantic inpainting tasks, either stand-alone or as initialization for non-parametric methods.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

page 12

research
03/27/2018

Structural inpainting

Scene-agnostic visual inpainting remains very challenging despite progre...
research
04/10/2017

Weakly-Supervised Spatial Context Networks

We explore the power of spatial context as a self-supervisory signal for...
research
03/16/2020

Pretraining Image Encoders without Reconstruction via Feature Prediction Loss

This work investigates three different loss functions for autoencoder-ba...
research
06/16/2020

Reconstruction of turbulent data with deep generative models for semantic inpainting from TURB-Rot database

We study the applicability of tools developed by the computer vision com...
research
01/17/2018

Light-weight pixel context encoders for image inpainting

In this work we propose Pixel Content Encoders (PCE), a light-weight ima...
research
03/11/2018

Cascade context encoder for improved inpainting

In this paper, we analyze if cascade usage of the context encoder with i...
research
12/03/2018

Chest X-Rays Image Inpainting with Context Encoders

Chest X-rays are one of the most commonly used technologies for medical ...

Please sign up or login with your details

Forgot password? Click here to reset