LAD: Language Augmented Diffusion for Reinforcement Learning

10/27/2022
by   Edwin Zhang, et al.
0

Learning skills from language provides a powerful avenue for generalization in reinforcement learning, although it remains a challenging task as it requires agents to capture the complex interdependencies between language, actions, and states. In this paper, we propose leveraging Language Augmented Diffusion models as a planner conditioned on language (LAD). We demonstrate the comparable performance of LAD with the state-of-the-art on the CALVIN language robotics benchmark with a much simpler architecture that contains no inductive biases specialized to robotics, achieving an average success rate (SR) of 72 compared to the best performance of 76 properties of language conditioned diffusion in reinforcement learning.

READ FULL TEXT
research
05/31/2022

IGLU Gridworld: Simple and Fast Environment for Embodied Dialog Agents

We present the IGLU Gridworld: a reinforcement learning environment for ...
research
04/26/2020

Reinforcement Learning Generalization with Surprise Minimization

Generalization remains a challenging problem for reinforcement learning ...
research
02/17/2023

Swapped goal-conditioned offline reinforcement learning

Offline goal-conditioned reinforcement learning (GCRL) can be challengin...
research
02/03/2023

AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners

Diffusion models have demonstrated their powerful generative capability ...
research
11/18/2022

Language-Conditioned Reinforcement Learning to Solve Misunderstandings with Action Corrections

Human-to-human conversation is not just talking and listening. It is an ...
research
07/21/2023

Selective Perception: Optimizing State Descriptions with Reinforcement Learning for Language Model Actors

Large language models (LLMs) are being applied as actors for sequential ...
research
04/27/2023

Learning a Diffusion Prior for NeRFs

Neural Radiance Fields (NeRFs) have emerged as a powerful neural 3D repr...

Please sign up or login with your details

Forgot password? Click here to reset