AI Chat AI Image Generator AI Video Text to Speech

VALAN: Vision and Language Agent Navigation

12/06/2019

∙

by Larry Lansing, et al.

∙

∙

VALAN is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture. The framework facilitates the development and evaluation of embodied agents for solving grounded language understanding tasks, such as Vision-and-Language Navigation and Vision-and-Dialog Navigation, in photo-realistic environments, such as Matterport3D and Google StreetView. We have added a minimal set of abstractions on top of SEED RL allowing us to generalize the architecture to solve a variety of other RL problems. In this article, we will describe VALAN's software abstraction and architecture, and also present an example of using VALAN to design agents for instruction-conditioned indoor navigation.

Larry Lansing
3 publications
Vihan Jain
14 publications
Harsh Mehta
13 publications
Haoshuo Huang
3 publications
Eugene Ie
20 publications

page 1

page 2

page 3

page 4

research

∙ 05/31/2022

IGLU Gridworld: Simple and Fast Environment for Embodied Dialog Agents

We present the IGLU Gridworld: a reinforcement learning environment for ...

0 Artem Zholus, et al. ∙

research

∙ 10/15/2019

SEED RL: Scalable and Efficient Deep-RL with Accelerated Central Inference

We present a modern scalable reinforcement learning agent called SEED (S...

0 Lasse Espeholt, et al. ∙

research

∙ 04/27/2022

Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learning

Building generalizable goal-conditioned agents from rich observations is...

0 Philippe Hansen-Estruch, et al. ∙

research

∙ 02/19/2019

DOM-Q-NET: Grounded RL on Structured Language

Building agents to interact with the web would allow for significant imp...

0 Sheng Jia, et al. ∙

research

∙ 02/27/2020

Review, Analyze, and Design a Comprehensive Deep Reinforcement Learning Framework

Reinforcement learning (RL) has emerged as a standard approach for build...

24 Ngoc Duy Nguyen, et al. ∙

research

∙ 04/15/2020

BabyAI++: Towards Grounded-Language Learning beyond Memorization

Despite success in many real-world tasks (e.g., robotics), reinforcement...

0 Tianshi Cao, et al. ∙

research

∙ 03/30/2021

Diagnosing Vision-and-Language Navigation: What Really Matters

Vision-and-language navigation (VLN) is a multimodal task where an agent...

10 Wanrong Zhu, et al. ∙