Guide Your Agent with Adaptive Multimodal Rewards

by   Changyeon Kim, et al.

Developing an agent capable of adapting to unseen environments remains a difficult challenge in imitation learning. In this work, we present Adaptive Return-conditioned Policy (ARP), an efficient framework designed to enhance the agent's generalization ability using natural language task descriptions and pre-trained multimodal encoders. Our key idea is to calculate a similarity between visual observations and natural language instructions in the pre-trained multimodal embedding space (such as CLIP) and use it as a reward signal. We then train a return-conditioned policy using expert demonstrations labeled with multimodal rewards. Because the multimodal rewards provide adaptive signals at each timestep, our ARP effectively mitigates the goal misgeneralization. This results in superior generalization performances even when faced with unseen text instructions, compared to existing text-conditioned policies. To improve the quality of rewards, we also introduce a fine-tuning method for pre-trained multimodal encoders, further enhancing the performance. Video demonstrations and source code are available on the project website:


page 6

page 7

page 8

page 9

page 21

page 22

page 23


Can Pre-Trained Text-to-Image Models Generate Visual Goals for Reinforcement Learning?

Pre-trained text-to-image generative models can produce diverse, semanti...

Instruction-Following Agents with Jointly Pre-Trained Vision-Language Models

Humans are excellent at understanding language and vision to accomplish ...

Controlled Text Generation with Natural Language Instructions

Large language models generate fluent texts and can follow natural langu...

Goal-conditioned Imitation Learning

Designing rewards for Reinforcement Learning (RL) is challenging because...

Improving Policy Learning via Language Dynamics Distillation

Recent work has shown that augmenting environments with language descrip...

LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action

Goal-conditioned policies for robotic navigation can be trained on large...

Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation

In the vision-and-language navigation (VLN) task, an agent follows natur...

Please sign up or login with your details

Forgot password? Click here to reset