DeepAI AI Chat
Log In Sign Up

Unsupervised Learning of Sensorimotor Affordances by Stochastic Future Prediction

by   Oleh Rybkin, et al.

Recently, much progress has been made building systems that can capture static image properties, but natural environments are intrinsically dynamic. For an intelligent agent, perception is responsible not only for capturing features of scene content, but also capturing its affordances: how the state of things can change, especially as the result of the agent's actions. We propose an unsupervised method to learn representations of the sensorimotor affordances of an environment. We do so by learning an embedding for stochastic future prediction that is (i) sensitive to scene dynamics and minimally sensitive to static scene content and (ii) compositional in nature, capturing the fact that changes in the environment can be composed to produce a cumulative change. We show that these two properties are sufficient to induce representations that are reusable across visually distinct scenes that share degrees of freedom. We show the applicability of our method to synthetic settings and its potential for understanding more complex, realistic visual settings.


page 7

page 14

page 15

page 16


Stochastic Video Prediction with Structure and Motion

While stochastic video prediction models enable future prediction under ...

dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learning

Learning graph representations is a fundamental task aimed at capturing ...

Trajectory Forecasting on Temporal Graphs

Predicting future locations of agents in the scene is an important probl...

Visual Attention in Imaginative Agents

We present a recurrent agent who perceives surroundings through a series...

Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks

We propose a framework for the completely unsupervised learning of laten...

Visual Reaction: Learning to Play Catch with Your Drone

In this paper we address the problem of visual reaction: the task of int...

Random Polyhedral Scenes: An Image Generator for Active Vision System Experiments

We present a Polyhedral Scene Generator system which creates a random sc...