Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution

03/26/2021
by   Chi Zhang, et al.
0

Spatial-temporal reasoning is a challenging task in Artificial Intelligence (AI) due to its demanding but unique nature: a theoretic requirement on representing and reasoning based on spatial-temporal knowledge in mind, and an applied requirement on a high-level cognitive system capable of navigating and acting in space and time. Recent works have focused on an abstract reasoning task of this kind – Raven's Progressive Matrices (RPM). Despite the encouraging progress on RPM that achieves human-level performance in terms of accuracy, modern approaches have neither a treatment of human-like reasoning on generalization, nor a potential to generate answers. To fill in this gap, we propose a neuro-symbolic Probabilistic Abduction and Execution (PrAE) learner; central to the PrAE learner is the process of probabilistic abduction and execution on a probabilistic scene representation, akin to the mental manipulation of objects. Specifically, we disentangle perception and reasoning from a monolithic model. The neural visual perception frontend predicts objects' attributes, later aggregated by a scene inference engine to produce a probabilistic scene representation. In the symbolic logical reasoning backend, the PrAE learner uses the representation to abduce the hidden rules. An answer is predicted by executing the rules on the probabilistic representation. The entire system is trained end-to-end in an analysis-by-synthesis manner without any visual attribute annotations. Extensive experiments demonstrate that the PrAE learner improves cross-configuration generalization and is capable of rendering an answer, in contrast to prior works that merely make a categorical choice from candidates.

READ FULL TEXT

page 2

page 4

research
11/29/2019

Learning Perceptual Inference by Contrasting

"Thinking in pictures," [1] i.e., spatial-temporal reasoning, effortless...
research
11/28/2022

Neuro-Symbolic Spatio-Temporal Reasoning

Knowledge about space and time is necessary to solve problems in the phy...
research
07/09/2020

Multi-Granularity Modularized Network for Abstract Visual Reasoning

Abstract visual reasoning connects mental abilities to the physical worl...
research
05/28/2019

Blocksworld Revisited: Learning and Reasoning to Generate Event-Sequences from Image Pairs

The process of identifying changes or transformations in a scene along w...
research
01/20/2020

A graph-based spatial temporal logic for knowledge representation and automated reasoning in cognitive robots

A new graph-based spatial temporal logic is proposed for knowledge repre...
research
08/12/2023

Learning Abstract Visual Reasoning via Task Decomposition: A Case Study in Raven Progressive Matrices

One of the challenges in learning to perform abstract reasoning is that ...

Please sign up or login with your details

Forgot password? Click here to reset