Log In Sign Up

PanGEA: The Panoramic Graph Environment Annotation Toolkit

by   Alexander Ku, et al.

PanGEA, the Panoramic Graph Environment Annotation toolkit, is a lightweight toolkit for collecting speech and text annotations in photo-realistic 3D environments. PanGEA immerses annotators in a web-based simulation and allows them to move around easily as they speak and/or listen. It includes database and cloud storage integration, plus utilities for automatically aligning recorded speech with manual transcriptions and the virtual pose of the annotators. Out of the box, PanGEA supports two tasks – collecting navigation instructions and navigation instruction following – and it could be easily adapted for annotating walking tours, finding and labeling landmarks or objects, and similar tasks. We share best practices learned from using PanGEA in a 20,000 hour annotation effort to collect the Room-Across-Room dataset. We hope that our open-source annotation toolkit and insights will both expedite future data collection efforts and spur innovation on the kinds of grounded language tasks such environments can support.


SEAN: Social Environment for Autonomous Navigation

Social navigation research is performed on a variety of robotic platform...

SANTLR: Speech Annotation Toolkit for Low Resource Languages

While low resource speech recognition has attracted a lot of attention f...

Less is More: Generating Grounded Navigation Instructions from Landmarks

We study the automatic generation of navigation instructions from 360-de...

Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding

We introduce Room-Across-Room (RxR), a new Vision-and-Language Navigatio...

Semi-automatic 3D Object Keypoint Annotation and Detection for the Masses

Creating computer vision datasets requires careful planning and lots of ...

Semantic Interior Mapology: A Toolbox For Indoor Scene Description From Architectural Floor Plans

We introduce the Semantic Interior Mapology (SIM) toolbox for the conver...

3D BAT: A Semi-Automatic, Web-based 3D Annotation Toolbox for Full-Surround, Multi-Modal Data Streams

In this paper, we focus on obtaining 2D and 3D labels, as well as track ...