Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation

by   Medhini Narasimhan, et al.

We introduce a learning-based approach for room navigation using semantic maps. Our proposed architecture learns to predict top-down belief maps of regions that lie beyond the agent's field of view while modeling architectural and stylistic regularities in houses. First, we train a model to generate amodal semantic top-down maps indicating beliefs of location, size, and shape of rooms by learning the underlying architectural patterns in houses. Next, we use these maps to predict a point that lies in the target room and train a policy to navigate to the point. We empirically demonstrate that by predicting semantic maps, the model learns common correlations found in houses and generalizes to novel environments. We also demonstrate that reducing the task of room navigation to point navigation improves the performance further.


page 1

page 5


Uncertainty-driven Planner for Exploration and Navigation

We consider the problems of exploration and point-goal navigation in pre...

Graph augmented Deep Reinforcement Learning in the GameRLand3D environment

We address planning and navigation in challenging 3D video games featuri...

Improving Vision-and-Language Navigation by Generating Future-View Image Semantics

Vision-and-Language Navigation (VLN) is the task that requires an agent ...

Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding

We introduce Room-Across-Room (RxR), a new Vision-and-Language Navigatio...

SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding

Semantic 2D maps are commonly used by humans and machines for navigation...

Learning to Move with Affordance Maps

The ability to autonomously explore and navigate a physical space is a f...

Predicting Dense and Context-aware Cost Maps for Semantic Robot Navigation

We investigate the task of object goal navigation in unknown environment...

Please sign up or login with your details

Forgot password? Click here to reset