DeepAI AI Chat
Log In Sign Up

Program-Guided Image Manipulators

by   Jiayuan Mao, et al.

Humans are capable of building holistic representations for images at various levels, from local objects, to pairwise relations, to global structures. The interpretation of structures involves reasoning over repetition and symmetry of the objects in the image. In this paper, we present the Program-Guided Image Manipulator (PG-IM), inducing neuro-symbolic program-like representations to represent and manipulate images. Given an image, PG-IM detects repeated patterns, induces symbolic programs, and manipulates the image using a neural network that is guided by the program. PG-IM learns from a single image, exploiting its internal statistics. Despite trained only on image inpainting, PG-IM is directly capable of extrapolation and regularity editing in a unified framework. Extensive experiments show that PG-IM achieves superior performance on all the tasks.


page 6

page 7

page 8

page 13

page 14

page 15

page 16

page 17


Multi-Plane Program Induction with 3D Box Priors

We consider two important aspects in understanding and editing images: m...

ProTo: Program-Guided Transformer for Program-Guided Tasks

Programs, consisting of semantic and structural information, play an imp...

Solving Visual Analogies Using Neural Algorithmic Reasoning

We consider a class of visual analogical reasoning problems that involve...

Perspective Plane Program Induction from a Single Image

We study the inverse graphics problem of inferring a holistic representa...

ImageEye: Batch Image Processing Using Program Synthesis

This paper presents a new synthesis-based approach for batch image proce...

Im2Struct: Recovering 3D Shape Structure from a Single RGB Image

We propose to recover 3D shape structures from single RGB images, where ...