Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View

01/10/2020
by   Harsh Mehta, et al.
9

The Touchdown dataset (Chen et al., 2019) provides instructions by human annotators for navigation through New York City streets and for resolving spatial descriptions at a given location. To enable the wider research community to work effectively with the Touchdown tasks, we are publicly releasing the 29k raw Street View panoramas needed for Touchdown. We follow the process used for the StreetLearn data release (Mirowski et al., 2019) to check panoramas for personally identifiable information and blur them as necessary. These have been added to the StreetLearn dataset and can be obtained via the same process as used previously for StreetLearn. We also provide a reference implementation for both of the Touchdown tasks: vision and language navigation (VLN) and spatial description resolution (SDR). We compare our model results to those given in Chen et al. (2019) and show that the panoramas we have added to StreetLearn fully support both Touchdown tasks and can be used effectively for further research and comparison.

READ FULL TEXT

page 2

page 3

page 4

research
03/04/2019

The StreetLearn Environment and Dataset

Navigation is a rich and well-grounded problem domain that drives progre...
research
07/12/2023

VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street View

Incremental decision making in real-world environments is one of the mos...
research
12/30/2020

Generating Landmark Navigation Instructions from Maps as a Graph-to-Text Problem

Car-focused navigation services are based on turns and distances of name...
research
05/19/2016

Stereotyping and Bias in the Flickr30K Dataset

An untested assumption behind the crowdsourced descriptions of the image...
research
03/01/2019

Learning To Follow Directions in Street View

Navigating and understanding the real world remains a key challenge in m...
research
11/30/2018

Efficient allocation of law enforcement resources using predictive police patrolling

Efficient allocation of scarce law enforcement resources is a hard probl...
research
10/31/2017

Updating the VESICLE-CNN Synapse Detector

We present an updated version of the VESICLE-CNN algorithm presented by ...

Please sign up or login with your details

Forgot password? Click here to reset