DeepAI
Log In Sign Up

Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration

02/21/2020
by   Cédric Colas, et al.
5

Autonomous reinforcement learning agents must be intrinsically motivated to explore their environment, discover potential goals, represent them and learn how to achieve them. As children do the same, they benefit from exposure to language, using it to formulate goals and imagine new ones as they learn their meaning. In our proposed learning architecture (IMAGINE), the agent freely explores its environment and turns natural language descriptions of interesting interactions from a social partner into potential goals. IMAGINE learns to represent goals by jointly learning a language model and a goal-conditioned reward function. Just like humans, our agent uses language compositionality to generate new goals by composing known ones. Leveraging modular model architectures based on Deep Sets and gated-attention mechanisms, IMAGINE autonomously builds a repertoire of behaviors and shows good zero-shot generalization properties for various types of generalization. When imagining its own goals, the agent leverages zero-shot generalization of the reward function to further train on imagined goals and refine its behavior. We present experiments in a simulated domain where the agent interacts with procedurally generated scenes containing objects of various types and colors, discovers goals, imagines others and learns to achieve them.

READ FULL TEXT

page 5

page 9

page 10

page 12

page 13

page 14

page 15

page 19

11/08/2019

Language Grounding through Social Interactions and Curiosity-Driven Multi-Goal Learning

Autonomous reinforcement learning agents, like children, do not have acc...
03/20/2020

Deep Sets for Generalization in RL

This paper investigates the idea of encoding object-centered representat...
02/15/2022

Interpretable Reinforcement Learning with Multilevel Subgoal Discovery

We propose a novel Reinforcement Learning model for discrete environment...
05/25/2022

Learning to Query Internet Text for Informing Reinforcement Learning Agents

Generalization to out of distribution tasks in reinforcement learning is...
10/07/2021

Situated Dialogue Learning through Procedural Environment Generation

We teach goal-driven agents to interactively act and speak in situated e...
12/03/2018

Generating Diverse Programs with Instruction Conditioned Reinforced Adversarial Learning

Advances in Deep Reinforcement Learning have led to agents that perform ...

Code Repositories

Imagine

Code for the paper Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration


view repo

playground_env

Implementation of the Playground environment from the paper Language as a Cognitive Tool to Imagine Goals inCuriosity-Driven Exploration.


view repo