Habitat: A Platform for Embodied AI Research

by   Manolis Savva, et al.

We present Habitat, a new platform for research in embodied artificial intelligence (AI). Habitat enables training embodied agents (virtual robots) in highly efficient photorealistic 3D simulation, before transferring the learned skills to reality. Specifically, Habitat consists of the following: 1. Habitat-Sim: a flexible, high-performance 3D simulator with configurable agents, multiple sensors, and generic 3D dataset handling (with built-in support for SUNCG, Matterport3D, Gibson datasets). Habitat-Sim is fast -- when rendering a scene from the Matterport3D dataset, Habitat-Sim achieves several thousand frames per second (fps) running single-threaded, and can reach over 10,000 fps multi-process on a single GPU, which is orders of magnitude faster than the closest simulator. 2. Habitat-API: a modular high-level library for end-to-end development of embodied AI algorithms -- defining embodied AI tasks (e.g. navigation, instruction following, question answering), configuring and training embodied agents (via imitation or reinforcement learning, or via classic SLAM), and benchmarking using standard metrics. These large-scale engineering contributions enable us to answer scientific questions requiring experiments that were till now impracticable or `merely' impractical. Specifically, in the context of point-goal navigation (1) we revisit the comparison between learning and SLAM approaches from two recent works and find evidence for the opposite conclusion -- that learning outperforms SLAM, if scaled to total experience far surpassing that of previous investigations, and (2) we conduct the first cross-dataset generalization experiments train, test x Matterport3D, Gibson for multiple sensors blind, RGB, RGBD, D and find that only agents with depth (D) sensors generalize across datasets. We hope that our open-source platform and these findings will advance research in embodied AI.


page 2

page 3

page 8

page 11

page 14

page 15

page 16


Polycraft World AI Lab (PAL): An Extensible Platform for Evaluating Artificial Intelligence Agents

As artificial intelligence research advances, the platforms used to eval...

ROS-X-Habitat: Bridging the ROS Ecosystem with Embodied AI

We introduce ROS-X-Habitat, a software interface that bridges the AI Hab...

The Replica Dataset: A Digital Replica of Indoor Spaces

We introduce Replica, a dataset of 18 highly photo-realistic 3D indoor s...

Large Batch Simulation for Deep Reinforcement Learning

We accelerate deep reinforcement learning-based training in visually com...

Emergence of Maps in the Memories of Blind Navigation Agents

Animal navigation research posits that organisms build and maintain inte...

MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence

We introduce MAgent, a platform to support research and development of m...

AllenAct: A Framework for Embodied AI Research

The domain of Embodied AI, in which agents learn to complete tasks throu...

Please sign up or login with your details

Forgot password? Click here to reset