DeepAI AI Chat
Log In Sign Up

Program-Guided Image Manipulators

09/04/2019
by   Jiayuan Mao, et al.
4

Humans are capable of building holistic representations for images at various levels, from local objects, to pairwise relations, to global structures. The interpretation of structures involves reasoning over repetition and symmetry of the objects in the image. In this paper, we present the Program-Guided Image Manipulator (PG-IM), inducing neuro-symbolic program-like representations to represent and manipulate images. Given an image, PG-IM detects repeated patterns, induces symbolic programs, and manipulates the image using a neural network that is guided by the program. PG-IM learns from a single image, exploiting its internal statistics. Despite trained only on image inpainting, PG-IM is directly capable of extrapolation and regularity editing in a unified framework. Extensive experiments show that PG-IM achieves superior performance on all the tasks.

READ FULL TEXT

page 6

page 7

page 8

page 13

page 14

page 15

page 16

page 17

11/19/2020

Multi-Plane Program Induction with 3D Box Priors

We consider two important aspects in understanding and editing images: m...
10/02/2021

ProTo: Program-Guided Transformer for Program-Guided Tasks

Programs, consisting of semantic and structural information, play an imp...
11/19/2021

Solving Visual Analogies Using Neural Algorithmic Reasoning

We consider a class of visual analogical reasoning problems that involve...
06/25/2020

Perspective Plane Program Induction from a Single Image

We study the inverse graphics problem of inferring a holistic representa...
04/06/2023

ImageEye: Batch Image Processing Using Program Synthesis

This paper presents a new synthesis-based approach for batch image proce...
04/16/2018

Im2Struct: Recovering 3D Shape Structure from a Single RGB Image

We propose to recover 3D shape structures from single RGB images, where ...