research
∙
08/19/2023
Skill Transformer: A Monolithic Policy for Mobile Manipulation
We present Skill Transformer, an approach for solving long-horizon robot...
share
research
∙
08/07/2023
Exploiting Generalization in Offline Reinforcement Learning via Unseen State Augmentations
Offline reinforcement learning (RL) methods strike a balance between exp...
research
∙
06/20/2023
HomeRobot: Open-Vocabulary Mobile Manipulation
HomeRobot (noun): An affordable compliant robot that navigates homes and...
research
∙
06/20/2023
Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation
We contribute the Habitat Synthetic Scene Dataset, a dataset of 211 high...
research
∙
06/13/2023
Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Second
We present Galactic, a large-scale simulation and reinforcement-learning...
research
∙
05/31/2023
Adaptive Coordination in Social Embodied Rearrangement
We present the task of "Social Rearrangement", consisting of cooperative...
research
∙
05/01/2023
IndoorSim-to-OutdoorReal: Learning to Navigate Outdoors without any Outdoor Experience
We present IndoorSim-to-OutdoorReal (I2O), an end-to-end learned visual ...
research
∙
04/03/2023
Navigating to Objects Specified by Images
Images are a convenient way to specify which particular object instance ...
research
∙
04/01/2023
Adaptive Skill Coordination for Robotic Mobile Manipulation
We present Adaptive Skill Coordination (ASC) - an approach for accomplis...
research
∙
03/31/2023
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?
We present the largest and most comprehensive empirical study of pre-tra...
research
∙
03/28/2023
BC-IRL: Learning Generalizable Reward Functions from Demonstrations
How well do reward functions learned with inverse reinforcement learning...
research
∙
03/14/2023
OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNav
We present a single neural network architecture composed of task-agnosti...
research
∙
01/30/2023
Emergence of Maps in the Memories of Blind Navigation Agents
Animal navigation research posits that organisms build and maintain inte...
research
∙
01/18/2023
PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNav
We study ObjectGoal Navigation - where a virtual robot situated in a new...
research
∙
12/02/2022
Navigating to Objects in the Real World
Semantic navigation is necessary to deploy mobile robots in uncontrolled...
research
∙
11/29/2022
Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instances
We consider the problem of embodied visual navigation given an image-goa...
research
∙
10/26/2022
ViNL: Visual Navigation and Locomotion Over Obstacles
We present Visual Navigation and Locomotion over obstacles (ViNL), which...
research
∙
10/11/2022
Habitat-Matterport 3D Semantics Dataset
We present the Habitat-Matterport 3D Semantics (HM3DSEM) dataset. HM3DSE...
research
∙
10/11/2022
VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangement
We present Variable Experience Rollout (VER), a technique for efficientl...
research
∙
07/21/2022
Rethinking Sim2Real: Lower Fidelity Simulation Leads to Higher Sim2Real Transfer in Navigation
If we want to train robots in simulation before deploying them in realit...
research
∙
06/24/2022
ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings
We present a scalable approach for learning open-world object-goal navig...
research
∙
06/16/2022
SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning
We introduce SoundSpaces 2.0, a platform for on-the-fly geometry-based a...
research
∙
06/02/2022
Is Mapping Necessary for Realistic PointGoal Navigation?
Can an autonomous agent navigate in a new environment without building a...
research
∙
05/22/2022
Housekeep: Tidying Virtual Households using Commonsense Reasoning
We introduce Housekeep, a benchmark to evaluate commonsense reasoning in...
research
∙
05/03/2022
Episodic Memory Question Answering
Egocentric augmented reality devices such as wearable glasses passively ...
research
∙
04/27/2022
Offline Visual Representation Learning for Embodied Navigation
How should we learn visual representations for embodied agents that must...
research
∙
04/07/2022
Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale
We present a large-scale study of imitating human demonstrations on task...
research
∙
04/06/2022
Simple and Effective Synthesis of Indoor 3D Scenes
We study the problem of synthesizing immersive 3D indoor scenes from one...
research
∙
10/27/2021
SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigation
Natural language instructions for visual navigation often use scene desc...
research
∙
10/05/2021
Waypoint Models for Instruction-guided Navigation in Continuous Environments
Little inquiry has explicitly addressed the role of action spaces in lan...
research
∙
09/22/2021
Learning Robust Agents for Visual Navigation in Dynamic Environments: The Winning Entry of iGibson Challenge 2021
This paper presents an approach for improving navigation in dynamic and ...
research
∙
09/17/2021
Realistic PointGoal Navigation via Auxiliary Losses and Information Bottleneck
We propose a novel architecture and training paradigm for training reali...
research
∙
09/16/2021
Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI
We present the Habitat-Matterport 3D (HM3D) dataset. HM3D is a large-sca...
research
∙
08/26/2021
The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation
It is fundamental for personal robots to reliably navigate to a specifie...
research
∙
06/28/2021
Habitat 2.0: Training Home Assistants to Rearrange their Habitat
We introduce Habitat 2.0 (H2.0), a simulation platform for training virt...
research
∙
06/26/2021
Model-Advantage Optimization for Model-Based Reinforcement Learning
Model-based Reinforcement Learning (MBRL) algorithms have been tradition...
research
∙
04/08/2021
Auxiliary Tasks and Exploration Enable ObjectNav
ObjectGoal Navigation (ObjectNav) is an embodied task wherein agents are...
research
∙
03/14/2021
Success Weighted by Completion Time: A Dynamics-Aware Evaluation Criteria for Embodied Navigation
We present Success weighted by Completion Time (SCT), a new metric for e...
research
∙
03/12/2021
Large Batch Simulation for Deep Reinforcement Learning
We accelerate deep reinforcement learning-based training in visually com...
research
∙
01/13/2021
Memory-Augmented Reinforcement Learning for Image-Goal Navigation
In this work, we address the problem of image-goal navigation in the con...
research
∙
12/11/2020
How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget
PointGoal navigation has seen significant recent interest and progress, ...
research
∙
11/24/2020
Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents
Deep reinforcement learning models are notoriously data hungry, yet real...
research
∙
11/24/2020
Learning Navigation Skills for Legged Robots with Learned Robot Embeddings
Navigation policies are commonly learned on idealized cylinder agents in...
research
∙
11/16/2020
Where Are You? Localization from Embodied Dialog
We present Where Are You? (WAY), a dataset of  6k dialogs in which two h...
research
∙
11/07/2020
Sim-to-Real Transfer for Vision-and-Language Navigation
We study the challenging problem of releasing a robot in a previously un...
research
∙
11/03/2020
Rearrangement: A Challenge for Embodied AI
We describe a framework for research and evaluation in Embodied AI. Our ...
research
∙
10/20/2020
SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency
Recent research in Visual Question Answering (VQA) has revealed state-of...
research
∙
10/13/2020
Contrast and Classify: Alternate Training for Robust VQA
Recent Visual Question Answering (VQA) models have shown impressive perf...
research
∙
10/02/2020
Semantic MapNet: Building Allocentric SemanticMaps and Representations from Egocentric Views
We study the task of semantic mapping - specifically, an embodied agent ...
research
∙
09/07/2020