DeepAI AI Chat
Log In Sign Up

Compositional Law Parsing with Latent Random Functions

by   Fan Shi, et al.
FUDAN University

Human cognition has compositionality. We understand a scene by decomposing the scene into different concepts (e.g. shape and position of an object) and learning the respective laws of these concepts which may be either natural (e.g. laws of motion) or man-made (e.g. laws of a game). The automatic parsing of these laws indicates the model's ability to understand the scene, which makes law parsing play a central role in many visual tasks. In this paper, we propose a deep latent variable model for Compositional LAw Parsing (CLAP). CLAP achieves the human-like compositionality ability through an encoding-decoding architecture to represent concepts in the scene as latent variables, and further employ concept-specific random functions, instantiated with Neural Processes, in the latent space to capture the law on each concept. Our experimental results demonstrate that CLAP outperforms the compared baseline methods in multiple visual tasks including intuitive physics, abstract visual reasoning, and scene representation. In addition, CLAP can learn concept-specific laws in a scene without supervision and one can edit laws through modifying the corresponding latent random functions, validating its interpretability and manipulability.


page 16

page 21

page 23

page 24

page 25

page 27

page 28

page 29


Raven's Progressive Matrices Completion with Latent Gaussian Process Priors

Abstract reasoning ability is fundamental to human intelligence. It enab...

The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision

We propose the Neuro-Symbolic Concept Learner (NS-CL), a model that lear...

Compositional Scene Representation Learning via Reconstruction: A Survey

Visual scene representation learning is an important research problem in...

Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations

Learning visual concepts from raw images without strong supervision is a...

Compression and the origins of Zipf's law for word frequencies

Here we sketch a new derivation of Zipf's law for word frequencies based...

SCAN: Learning Abstract Hierarchical Compositional Visual Concepts

The natural world is infinitely diverse, yet this diversity arises from ...

Compositional Visual Generation and Inference with Energy Based Models

A vital aspect of human intelligence is the ability to compose increasin...