Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI

by   Santhosh K. Ramakrishnan, et al.

We present the Habitat-Matterport 3D (HM3D) dataset. HM3D is a large-scale dataset of 1,000 building-scale 3D reconstructions from a diverse set of real-world locations. Each scene in the dataset consists of a textured 3D mesh reconstruction of interiors such as multi-floor residences, stores, and other private indoor spaces. HM3D surpasses existing datasets available for academic research in terms of physical scale, completeness of the reconstruction, and visual fidelity. HM3D contains 112.5k m^2 of navigable space, which is 1.4 - 3.7x larger than other building-scale datasets such as MP3D and Gibson. When compared to existing photorealistic 3D datasets such as Replica, MP3D, Gibson, and ScanNet, images rendered from HM3D have 20 - 85 images captured with real cameras, and HM3D meshes have 34 - 91 artifacts due to incomplete surface reconstruction. The increased scale, fidelity, and diversity of HM3D directly impacts the performance of embodied AI agents trained using it. In fact, we find that HM3D is `pareto optimal' in the following sense – agents trained to perform PointGoal navigation on HM3D achieve the highest performance regardless of whether they are evaluated on HM3D, Gibson, or MP3D. No similar claim can be made about training on other datasets. HM3D-trained PointNav agents achieve 100 retire that episode dataset.



There are no comments yet.


page 1

page 3

page 6

page 7

page 17

page 18

page 20

page 21


The Replica Dataset: A Digital Replica of Indoor Spaces

We introduce Replica, a dataset of 18 highly photo-realistic 3D indoor s...

Joint 2D-3D-Semantic Data for Indoor Scene Understanding

We present a dataset of large-scale indoor spaces that provides a variet...

IRS: A Large Synthetic Indoor Robotics Stereo Dataset for Disparity and Surface Normal Estimation

Indoor robotics localization, navigation and interaction heavily rely on...

Matterport3D: Learning from RGB-D Data in Indoor Environments

Access to large, diverse RGB-D datasets is critical for training RGB-D s...

PetroSurf3D - A Dataset for high-resolution 3D Surface Segmentation

The development of powerful 3D scanning hardware and reconstruction algo...

MVB: A Large-Scale Dataset for Baggage Re-Identification and Merged Siamese Networks

In this paper, we present a novel dataset named MVB (Multi View Baggage)...

A Unified View-Graph Selection Framework for Structure from Motion

View-graph is an essential input to large-scale structure from motion (S...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.