Stefan Lee

Audio-Visual Scene-Aware Dialog

We introduce the task of scene-aware dialog. Given a follow-up question ...
01/25/2019 ∙ by Huda Alamri, et al. ∙ 48 ∙ share

read it
nocaps: novel object captioning at scale

Image captioning models have achieved impressive results on datasets con...
12/20/2018 ∙ by Harsh Agrawal, et al. ∙ 46 ∙ share

read it
Decentralized Distributed PPO: Solving PointGoal Navigation

We present Decentralized Distributed Proximal Policy Optimization (DD-PP...
11/01/2019 ∙ by Erik Wijmans, et al. ∙ 45 ∙ share

read it
Are We Making Real Progress in Simulated Environments? Measuring the Sim2Real Gap in Embodied Visual Navigation

Does progress in simulation translate to progress in robotics? Specifica...
12/13/2019 ∙ by Abhishek Kadian, et al. ∙ 37 ∙ share

read it
Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

Many vision and language models suffer from poor visual grounding - ofte...
02/11/2019 ∙ by Ramprasaath R. Selvaraju, et al. ∙ 26 ∙ share

read it
EvalAI: Towards Better Evaluation Systems for AI Agents

We introduce EvalAI, an open source platform for evaluating and comparin...
02/10/2019 ∙ by Deshraj Yadav, et al. ∙ 24 ∙ share

read it
Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering

We propose a new class of probabilistic neural-symbolic models, that hav...
02/21/2019 ∙ by Ramakrishna Vedantam, et al. ∙ 24 ∙ share

read it
Counterfactual Visual Explanations

A counterfactual query is typically of the form 'For situation X, why wa...
04/16/2019 ∙ by Yash Goyal, et al. ∙ 24 ∙ share

read it
12-in-1: Multi-Task Vision and Language Representation Learning

Much of vision-and-language research focuses on a small but diverse set ...
12/05/2019 ∙ by Jiasen Lu, et al. ∙ 22 ∙ share

read it
Visual Curiosity: Learning to Ask Questions to Learn Visual Recognition

In an open-world setting, it is inevitable that an intelligent agent (e....
10/01/2018 ∙ by Jianwei Yang, et al. ∙ 20 ∙ share

read it
Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance

Individual neurons in convolutional neural networks supervised for image...
08/08/2018 ∙ by Ramprasaath R. Selvaraju, et al. ∙ 16 ∙ share

read it
Embodied Question Answering in Photorealistic Environments with Point Cloud Perception

To help bridge the gap between internet vision-style problems and the go...
04/06/2019 ∙ by Erik Wijmans, et al. ∙ 12 ∙ share

read it
Emergence of Compositional Language with Deep Generational Transmission

Consider a collaborative task that requires communication. Two agents ar...
04/19/2019 ∙ by Michael Cogswell, et al. ∙ 12 ∙ share

read it
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks

We present ViLBERT (short for Vision-and-Language BERT), a model for lea...
08/06/2019 ∙ by Jiasen Lu, et al. ∙ 9 ∙ share

read it
Learn from Your Neighbor: Learning Multi-modal Mappings from Sparse Annotations

Many structured prediction problems (particularly in vision and language...
06/08/2018 ∙ by Ashwin Kalyan, et al. ∙ 8 ∙ share

read it
Graph R-CNN for Scene Graph Generation

We propose a novel scene graph generation model called Graph R-CNN, that...
08/01/2018 ∙ by Jianwei Yang, et al. ∙ 6 ∙ share

read it
Neural Modular Control for Embodied Question Answering

We present a modular approach for learning policies for navigation over ...
10/26/2018 ∙ by Abhishek Das, et al. ∙ 6 ∙ share

read it
Chasing Ghosts: Instruction Following as Bayesian State Tracking

A visually-grounded navigation instruction can be interpreted as a seque...
07/03/2019 ∙ by Peter Anderson, et al. ∙ 3 ∙ share

read it
Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks

Convolutional Neural Networks have achieved state-of-the-art performance...
11/19/2015 ∙ by Stefan Lee, et al. ∙ 0 ∙ share

read it
Bidirectional Beam Search: Forward-Backward Inference in Neural Sequence Models for Fill-in-the-Blank Image Captioning

We develop the first approximate inference algorithm for 1-Best (and M-B...
05/24/2017 ∙ by Qing Sun, et al. ∙ 0 ∙ share

read it
Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog

A number of recent works have proposed techniques for end-to-end learnin...
06/26/2017 ∙ by Satwik Kottur, et al. ∙ 0 ∙ share

read it
The Promise of Premise: Harnessing Question Premises in Visual Question Answering

In this paper, we make a simple observation that questions about images ...
05/01/2017 ∙ by Aroma Mahendru, et al. ∙ 0 ∙ share

read it
Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning

We introduce the first goal-driven training for visual question answerin...
03/20/2017 ∙ by Abhishek Das, et al. ∙ 0 ∙ share

read it
Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models

Neural sequence models are widely used to model time-series data in many...
10/07/2016 ∙ by Ashwin K Vijayakumar, et al. ∙ 0 ∙ share

read it
Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles

Many practical perception systems exist within larger processes that inc...
06/24/2016 ∙ by Stefan Lee, et al. ∙ 0 ∙ share

read it
Evaluating Visual Conversational Agents via Cooperative Human-AI Games

As AI continues to advance, human-AI teams are inevitable. However, prog...
08/17/2017 ∙ by Prithvijit Chattopadhyay, et al. ∙ 0 ∙ share

read it
Embodied Question Answering

We present a new AI task -- Embodied Question Answering (EmbodiedQA) -- ...
11/30/2017 ∙ by Abhishek Das, et al. ∙ 0 ∙ share

read it
Overcoming Language Priors in Visual Question Answering with Adversarial Regularization

Modern Visual Question Answering (VQA) models have been shown to rely he...
10/08/2018 ∙ by Sainandan Ramakrishnan, et al. ∙ 0 ∙ share

read it
Sunny and Dark Outside?! Improving Answer Consistency in VQA through Entailed Question Generation

While models for Visual Question Answering (VQA) have steadily improved ...
09/10/2019 ∙ by Arijit Ray, et al. ∙ 0 ∙ share

read it
Question-Conditioned Counterfactual Image Generation for VQA

While Visual Question Answering (VQA) models continue to push the state-...
11/14/2019 ∙ by Jingjing Pan, et al. ∙ 0 ∙ share

read it

Stefan Lee

Featured Co-authors