DeepAI AI Chat
Log In Sign Up

Reusable Slotwise Mechanisms

by   Trang Nguyen, et al.

Agents that can understand and reason over the dynamics of objects can have a better capability to act robustly and generalize to novel scenarios. Such an ability, however, requires a suitable representation of the scene as well as an understanding of the mechanisms that govern the interactions of different subsets of objects. To address this problem, we propose RSM, or Reusable Slotwise Mechanisms, that jointly learns a slotwise representation of the scene and a modular architecture that dynamically chooses one mechanism among a set of reusable mechanisms to predict the next state of each slot. RSM crucially takes advantage of a Central Contextual Information (CCI), which lets each selected reusable mechanism access the rest of the slots through a bottleneck, effectively allowing for modeling higher order and complex interactions that might require a sparse subset of objects. We show how this model outperforms state-of-the-art methods in a variety of next-step prediction tasks ranging from grid-world environments to Atari 2600 games. Particularly, we challenge methods that model the dynamics with Graph Neural Networks (GNNs) on top of slotwise representations, and modular architectures that restrict the interactions to be only pairwise. Finally, we show that RSM is able to generalize to scenes with objects varying in number and shape, highlighting its out-of-distribution generalization capabilities. Our implementation is available online[].


page 7

page 8

page 14

page 16

page 17

page 18

page 19

page 20


Stochastic Occupancy Grid Map Prediction in Dynamic Scenes

This paper presents two variations of a novel stochastic prediction algo...

Simplicial Attention Networks

Graph representation learning methods have mostly been limited to the mo...

Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects

Capturing scene dynamics and predicting the future scene state is challe...

GraphMapper: Efficient Visual Navigation by Scene Graph Generation

Understanding the geometric relationships between objects in a scene is ...

Solving Reasoning Tasks with a Slot Transformer

The ability to carve the world into useful abstractions in order to reas...

Recurrent Independent Mechanisms

Learning modular structures which reflect the dynamics of the environmen...