Topological Planning with Transformers for Vision-and-Language Navigation

12/09/2020
by   Kevin Chen, et al.
1

Conventional approaches to vision-and-language navigation (VLN) are trained end-to-end but struggle to perform well in freely traversable environments. Inspired by the robotics community, we propose a modular approach to VLN using topological maps. Given a natural language instruction and topological map, our approach leverages attention mechanisms to predict a navigation plan in the map. The plan is then executed with low-level actions (e.g. forward, rotate) using a robust controller. Experiments show that our method outperforms previous end-to-end approaches, generates interpretable navigation plans, and exhibits intelligent behaviors such as backtracking.

READ FULL TEXT

page 5

page 8

page 12

page 13

page 14

page 19

page 20

page 21

research
09/24/2018

Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation

We propose an end-to-end deep learning model for translating free-form n...
research
10/16/2021

Lifelong Topological Visual Navigation

The ability for a robot to navigate with only the use of vision is appea...
research
07/24/2023

GridMM: Grid Memory Map for Vision-and-Language Navigation

Vision-and-language navigation (VLN) enables the agent to navigate to a ...
research
03/10/2022

Cross-modal Map Learning for Vision and Language Navigation

We consider the problem of Vision-and-Language Navigation (VLN). The maj...
research
06/07/2021

Hierarchical Robot Navigation in Novel Environments using Rough 2-D Maps

In robot navigation, generalizing quickly to unseen environments is esse...
research
03/07/2022

Find a Way Forward: a Language-Guided Semantic Map Navigator

This paper attacks the problem of language-guided navigation in a new pe...
research
02/19/2021

Learning Composable Behavior Embeddings for Long-horizon Visual Navigation

Learning high-level navigation behaviors has important implications: it ...

Please sign up or login with your details

Forgot password? Click here to reset