Smart, Sparse Contours to Represent and Edit Images

by   Tali Dekel, et al.

We study the problem of reconstructing an image from information stored at sparse contour locations. Existing contour-based image reconstruction methods struggle to balance contour sparsity and reconstruction fidelity. Therefore, denser contours are needed to capture subtle texture information even though contours were not meant for textures. We propose a novel image representation where image content is characterized by contours with gradient information via an encoder-decoder network, while image details are modeled by a conditional generative adversarial network. We show that high-quality reconstructions with high fidelity to the source image can be obtained from extremely sparse input, e.g., comprising less than 6 details and fine structures in regions where no input information is provided. The semantic knowledge encoded into our model and the sparsity of the input allows using contours as an intuitive interface for semantically-aware image manipulation: local edits in contour domain such as scaling, translation and erasing, translate to long-range and coherent changes in the pixel space. Experiments on a variety of datasets verify the versatility and convenience afforded by our models.



There are no comments yet.


page 1

page 2

page 4

page 5

page 6

page 7

page 8


PI-REC: Progressive Image Reconstruction Network With Edge and Color Domain

We propose a universal image reconstruction method to represent detailed...

Mask Iterative Hard Thresholding Algorithms for Sparse Image Reconstruction of Objects with Known Contour

We develop mask iterative hard thresholding algorithms (mask IHT and mas...

Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

A lot of work has been done towards reconstructing the 3D facial structu...

Neural Photo Editing with Introspective Adversarial Networks

The increasingly photorealistic sample quality of generative image model...

Adversarial Colorization Of Icons Based On Structure And Color Conditions

We present a system to help designers create icons that are widely used ...

Walk the Lines: Object Contour Tracing CNN for Contour Completion of Ships

We develop a new contour tracing algorithm to enhance the results of the...

Accurate and Fast reconstruction of Porous Media from Extremely Limited Information Using Conditional Generative Adversarial Network

Porous media are ubiquitous in both nature and engineering applications,...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.