b'Dhruv Batra'

research

∙ 05/22/2022

Housekeep: Tidying Virtual Households using Commonsense Reasoning

We introduce Housekeep, a benchmark to evaluate commonsense reasoning in...

Yash Kant, et al. ∙

research

∙ 05/03/2022

Episodic Memory Question Answering

Egocentric augmented reality devices such as wearable glasses passively ...

Samyak Datta, et al. ∙

research

∙ 04/27/2022

Offline Visual Representation Learning for Embodied Navigation

How should we learn visual representations for embodied agents that must...

Karmesh Yadav, et al. ∙

research

∙ 04/07/2022

Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale

We present a large-scale study of imitating human demonstrations on task...

Ram Ramrakhya, et al. ∙

research

∙ 04/06/2022

Simple and Effective Synthesis of Indoor 3D Scenes

We study the problem of synthesizing immersive 3D indoor scenes from one...

Jing Yu Koh, et al. ∙

research

∙ 10/27/2021

SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigation

Natural language instructions for visual navigation often use scene desc...

Abhinav Moudgil, et al. ∙

research

∙ 10/05/2021

Waypoint Models for Instruction-guided Navigation in Continuous Environments

Little inquiry has explicitly addressed the role of action spaces in lan...

Jacob Krantz, et al. ∙

research

∙ 09/22/2021

Learning Robust Agents for Visual Navigation in Dynamic Environments: The Winning Entry of iGibson Challenge 2021

This paper presents an approach for improving navigation in dynamic and ...

Naoki Yokoyama, et al. ∙

research

∙ 09/17/2021

Realistic PointGoal Navigation via Auxiliary Losses and Information Bottleneck

We propose a novel architecture and training paradigm for training reali...

Guillermo Grande, et al. ∙

research

∙ 09/16/2021

Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI

We present the Habitat-Matterport 3D (HM3D) dataset. HM3D is a large-sca...

Santhosh K. Ramakrishnan, et al. ∙

research

∙ 08/26/2021

The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation

It is fundamental for personal robots to reliably navigate to a specifie...

Xiaoming Zhao, et al. ∙

research

∙ 06/28/2021

Habitat 2.0: Training Home Assistants to Rearrange their Habitat

We introduce Habitat 2.0 (H2.0), a simulation platform for training virt...

Andrew Szot, et al. ∙

research

∙ 06/26/2021

Model-Advantage Optimization for Model-Based Reinforcement Learning

Model-based Reinforcement Learning (MBRL) algorithms have been tradition...

Nirbhay Modhe, et al. ∙

research

∙ 04/08/2021

Auxiliary Tasks and Exploration Enable ObjectNav

ObjectGoal Navigation (ObjectNav) is an embodied task wherein agents are...

Joel Ye, et al. ∙

research

∙ 03/14/2021

Success Weighted by Completion Time: A Dynamics-Aware Evaluation Criteria for Embodied Navigation

We present Success weighted by Completion Time (SCT), a new metric for e...

Naoki Yokoyama, et al. ∙

research

∙ 03/12/2021

Large Batch Simulation for Deep Reinforcement Learning

We accelerate deep reinforcement learning-based training in visually com...

Brennan Shacklett, et al. ∙

research

∙ 01/13/2021

Memory-Augmented Reinforcement Learning for Image-Goal Navigation

In this work, we address the problem of image-goal navigation in the con...

Lina Mezghani, et al. ∙

research

∙ 12/11/2020

How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget

PointGoal navigation has seen significant recent interest and progress, ...

Erik Wijmans, et al. ∙

research

∙ 11/24/2020

Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents

Deep reinforcement learning models are notoriously data hungry, yet real...

Joanne Truong, et al. ∙

research

∙ 11/24/2020

Learning Navigation Skills for Legged Robots with Learned Robot Embeddings

Navigation policies are commonly learned on idealized cylinder agents in...

Joanne Truong, et al. ∙

research

∙ 11/16/2020

Where Are You? Localization from Embodied Dialog

We present Where Are You? (WAY), a dataset of 6k dialogs in which two h...

Meera Hahn, et al. ∙

research

∙ 11/07/2020

Sim-to-Real Transfer for Vision-and-Language Navigation

We study the challenging problem of releasing a robot in a previously un...

Peter Anderson, et al. ∙

research

∙ 11/03/2020

Rearrangement: A Challenge for Embodied AI

We describe a framework for research and evaluation in Embodied AI. Our ...

Dhruv Batra, et al. ∙

research

∙ 10/20/2020

SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency

Recent research in Visual Question Answering (VQA) has revealed state-of...

Sameer Dharur, et al. ∙

research

∙ 10/13/2020

Contrast and Classify: Alternate Training for Robust VQA

Recent Visual Question Answering (VQA) models have shown impressive perf...

Yash Kant, et al. ∙

research

∙ 10/02/2020

Semantic MapNet: Building Allocentric SemanticMaps and Representations from Egocentric Views

We study the task of semantic mapping - specifically, an embodied agent ...

Vincent Cartillier, et al. ∙

research

∙ 09/07/2020

Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents

Recent work has presented embodied agents that can navigate to point-goa...

Samyak Datta, et al. ∙

research

∙ 07/24/2020

Dialog without Dialog Data: Learning Visual Dialog Agents from VQA Data

Can we develop visually grounded dialog agents that can efficiently adap...

Michael Cogswell, et al. ∙

research

∙ 07/23/2020

Spatially Aware Multimodal Transformers for TextVQA

Textual cues are essential for everyday tasks like buying groceries and ...

Yash Kant, et al. ∙

research

∙ 07/20/2020

Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation

We introduce a learning-based approach for room navigation using semanti...

Medhini Narasimhan, et al. ∙

research

∙ 07/09/2020

Auxiliary Tasks Speed Up Learning PointGoal Navigation

PointGoal Navigation is an embodied task that requires agents to navigat...

Joel Ye, et al. ∙

research

∙ 06/23/2020

ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects

We revisit the problem of Object-Goal Navigation (ObjectNav). In its sim...

Dhruv Batra, et al. ∙

research

∙ 04/30/2020

Improving Vision-and-Language Navigation with Image-Text Pairs from the Web

Following a navigation instruction such as 'Walk down the stairs and sto...

Arjun Majumdar, et al. ∙

research

∙ 04/06/2020

Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments

We develop a language-guided navigation task set in a continuous 3D envi...

Jacob Krantz, et al. ∙

research

∙ 03/12/2020

Analyzing Visual Representations in Embodied Navigation Tasks

Recent advances in deep reinforcement learning require a large amount of...

Erik Wijmans, et al. ∙

research

∙ 12/13/2019

Are We Making Real Progress in Simulated Environments? Measuring the Sim2Real Gap in Embodied Visual Navigation

Does progress in simulation translate to progress in robotics? Specifica...

Abhishek Kadian, et al. ∙

research

∙ 12/05/2019

Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline

Prior work in visual dialog has focused on training deep neural models o...

Vishvak Murahari, et al. ∙

research

∙ 11/01/2019

Decentralized Distributed PPO: Solving PointGoal Navigation

We present Decentralized Distributed Proximal Policy Optimization (DD-PP...

Erik Wijmans, et al. ∙

research

∙ 09/23/2019

Improving Generative Visual Dialog by Answering Diverse Questions

Prior work on training generative Visual Dialog models with reinforcemen...

Vishvak Murahari, et al. ∙

research

∙ 08/22/2019

Sequential Latent Spaces for Modeling the Intention During Diverse Image Captioning

Diverse and accurate vision+language modeling is an important goal to re...

Jyoti Aneja, et al. ∙

research

∙ 08/06/2019

ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks

We present ViLBERT (short for Vision-and-Language BERT), a model for lea...

Jiasen Lu, et al. ∙

research

∙ 07/24/2019

Unsupervised Discovery of Decision States for Transfer in Reinforcement Learning

We present a hierarchical reinforcement learning (HRL) or options framew...

Nirbhay Modhe, et al. ∙

research

∙ 07/03/2019

Chasing Ghosts: Instruction Following as Bayesian State Tracking

A visually-grounded navigation instruction can be interpreted as a seque...

Peter Anderson, et al. ∙

research

∙ 06/13/2019

The Replica Dataset: A Digital Replica of Indoor Spaces

We introduce Replica, a dataset of 18 highly photo-realistic 3D indoor s...

Julian Straub, et al. ∙

research

∙ 05/18/2019

SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigation

We propose SplitNet, a method for decoupling visual perception and polic...

Daniel Gordon, et al. ∙

research

∙ 04/19/2019

Emergence of Compositional Language with Deep Generational Transmission

Consider a collaborative task that requires communication. Two agents ar...

Michael Cogswell, et al. ∙

research

∙ 04/18/2019

Towards VQA Models that can Read

Studies have shown that a dominant class of questions asked by visually ...

Amanpreet Singh, et al. ∙

research

∙ 04/16/2019

Counterfactual Visual Explanations

A counterfactual query is typically of the form 'For situation X, why wa...

Yash Goyal, et al. ∙

research

∙ 04/09/2019

Multi-Target Embodied Question Answering

Embodied Question Answering (EQA) is a relatively new task where an agen...

Licheng Yu, et al. ∙

research

∙ 04/09/2019

Embodied Visual Recognition

Passive visual systems typically fail to recognize objects in the amodal...

Jianwei Yang, et al. ∙

Dhruv Batra

Featured Co-authors