Guaranteed Discovery of Controllable Latent States with Multi-Step Inverse Models

07/17/2022
by   Alex Lamb, et al.
17

A person walking along a city street who tries to model all aspects of the world would quickly be overwhelmed by a multitude of shops, cars, and people moving in and out of view, following their own complex and inscrutable dynamics. Exploration and navigation in such an environment is an everyday task, requiring no vast exertion of mental resources. Is it possible to turn this fire hose of sensory information into a minimal latent state which is necessary and sufficient for an agent to successfully act in the world? We formulate this question concretely, and propose the Agent-Controllable State Discovery algorithm (AC-State), which has theoretical guarantees and is practically demonstrated to discover the minimal controllable latent state which contains all of the information necessary for controlling the agent, while fully discarding all irrelevant information. This algorithm consists of a multi-step inverse model (predicting actions from distant observations) with an information bottleneck. AC-State enables localization, exploration, and navigation without reward or demonstrations. We demonstrate the discovery of controllable latent state in three domains: localizing a robot arm with distractions (e.g., changing lighting conditions and background), exploring in a maze alongside other agents, and navigating in the Matterport house simulator.

READ FULL TEXT

page 6

page 7

page 8

page 9

research
08/25/2020

ICE-Talk: an Interface for a Controllable Expressive Talking Machine

ICE-Talk is an open source web-based GUI that allows the use of a TTS sy...
research
11/05/2018

Contingency-Aware Exploration in Reinforcement Learning

This paper investigates whether learning contingency-awareness and contr...
research
05/27/2022

Isolating and Leveraging Controllable and Noncontrollable Visual Dynamics in World Models

World models learn the consequences of actions in vision-based interacti...
research
03/27/2023

Model-Based Reinforcement Learning with Isolated Imaginations

World models learn the consequences of actions in vision-based interacti...
research
12/15/2021

Towards Controllable Agent in MOBA Games with Generative Modeling

We propose novel methods to develop action controllable agent that behav...
research
07/02/2018

Perfectly Controllable Multi-Agent Networks

This note investigates how to design topology structures to ensure the c...
research
08/03/2017

Independently Controllable Factors

It has been postulated that a good representation is one that disentangl...

Please sign up or login with your details

Forgot password? Click here to reset