Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems

06/29/2020
by   Anirudh Goyal, et al.
0

Modeling a structured, dynamic environment like a video game requires keeping track of the objects and their states (declarative knowledge) as well as predicting how objects behave (procedural knowledge). Black-box models with a monolithic hidden state often lack systematicity: they fail to apply procedural knowledge consistently and uniformly. For example, in a video game, correct prediction of one enemy's trajectory does not ensure correct prediction of another's. We address this issue via an architecture that factorizes declarative and procedural knowledge and that imposes modularity within each form of knowledge. The architecture consists of active modules called object files that maintain the state of a single object and invoke passive external knowledge sources called schemata that prescribe state updates. To use a video game as an illustration, two enemies of the same type will share schemata but will each have their own object file to encode their distinct state (e.g., health, position). We propose to use attention to control the determination of which object files to update, the selection of schemata, and the propagation of information between object files. The resulting architecture is a drop-in replacement conforming to the same input-output interface as normal recurrent networks (e.g., LSTM, GRU) yet achieves substantially better generalization on environments that have factorized declarative and procedural knowledge, including a challenging intuitive physics benchmark.

READ FULL TEXT

page 2

page 20

research
08/21/2019

Towards Marrying Files to Objects

To deal with the constant growth of unstructured data, vendors have depl...
research
05/15/2020

An Object Model for the Representation of Empirical Knowledge

We are currently designing an object oriented model which describes stat...
research
02/25/2021

Checkpointing with cp: the POSIX Shared Memory System

We present the checkpointing scheme of Abacus, an N-body simulation code...
research
05/13/2019

Joint Object and State Recognition using Language Knowledge

The state of an object is an important piece of knowledge in robotics ap...
research
03/03/2021

Exploitation and Sanitization of Hidden Data in PDF Files

Organizations publish and share more and more electronic documents like ...
research
08/18/2020

TIDE: A General Toolbox for Identifying Object Detection Errors

We introduce TIDE, a framework and associated toolbox for analyzing the ...

Please sign up or login with your details

Forgot password? Click here to reset