research
∙
09/06/2023
A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models
Key to tasks that require reasoning about natural language in visual con...
share
research
∙
07/19/2023
IncDSI: Incrementally Updatable Document Retrieval
Differentiable Search Index is a recently proposed paradigm for document...
research
∙
05/21/2023
Continually Improving Extractive QA via Human Feedback
We study continually improving an extractive question answering (QA) sys...
research
∙
05/11/2023
Semantic uncertainty guides the extension of conventions to new referents
A long tradition of studies in psycholinguistics has examined the format...
research
∙
03/14/2023
CB2: Collaborative Natural Language Interaction Research Platform
CB2 is a multi-agent platform to study collaborative natural language in...
research
∙
12/19/2022
Continual Learning for Instruction Following from Realtime Feedback
We study the problem of continually training an instruction-following ag...
research
∙
11/29/2022
Abstract Visual Reasoning with Tangram Shapes
We introduce KiloGram, a resource for studying abstract visual reasoning...
research
∙
11/03/2022
lilGym: Natural Language Visual Reasoning with Reinforcement Learning
We present lilGym, a new benchmark for language-conditioned reinforcemen...
research
∙
05/02/2022
Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages
We introduce Wav2Seq, the first self-supervised approach to pre-train bo...
research
∙
03/18/2022
Simulating Bandit Learning from User Feedback for Extractive Question Answering
We study learning from user feedback for extractive question answering b...
research
∙
11/19/2021
SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech
Progress in speech processing has been facilitated by shared datasets an...
research
∙
09/28/2021
When in Doubt: Improving Classification Performance with Alternating Normalization
We introduce Classification with Alternating Normalization (CAN), a non-...
research
∙
09/14/2021
Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition
This paper is a study of performance-efficiency trade-offs in pre-traine...
research
∙
09/09/2021
Analysis of Language Change in Collaborative Instruction Following
We analyze language change over time in a collaborative, goal-oriented i...
research
∙
08/16/2021
Who's Waldo? Linking People Across Text and Images
We present a task and benchmark dataset for person-centric visual ground...
research
∙
08/10/2021
Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior
We study continual learning for natural language instruction generation,...
research
∙
07/12/2021
A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution
Natural language provides an accessible and expressive interface to spec...
research
∙
11/14/2020
Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following
We study the problem of learning a robot policy to follow natural langua...
research
∙
06/10/2020
Revisiting Few-sample BERT Fine-tuning
We study the problem of few-sample fine-tuning of BERT contextual repres...
research
∙
05/04/2020
What is Learned in Visually Grounded Neural Syntax Acquisition
Visual features are a promising signal for learning bootstrap textual mo...
research
∙
04/06/2020
Evaluating NLP Models via Contrast Sets
Standard test sets for supervised learning evaluate in-distribution gene...
research
∙
01/10/2020
Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View
The Touchdown dataset (Chen et al., 2019) provides instructions by human...
research
∙
11/09/2019
Interactive Classification by Asking Informative Questions
Natural language systems often rely on a single, potentially ambiguous i...
research
∙
10/21/2019
Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight
We propose a joint simulation and real-world learning framework for mapp...
research
∙
10/08/2019
Executing Instructions in Situated Collaborative Interactions
We study a collaborative scenario where a user not only instructs a syst...
research
∙
09/23/2019
NLVR2 Visual Bias Analysis
NLVR2 (Suhr et al., 2019) was designed to be robust for language bias th...
research
∙
04/21/2019
BERTScore: Evaluating Text Generation with BERT
We propose BERTScore, an automatic evaluation metric for text generation...
research
∙
11/29/2018
Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
We study the problem of jointly reasoning about language and vision thro...
research
∙
11/21/2018
Early Fusion for Goal Directed Robotic Vision
Increasingly, perceptual systems are being codified as strict pipelines ...
research
∙
11/10/2018
Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction
We propose an approach for mapping natural language instructions and raw...
research
∙
11/01/2018
A Corpus for Reasoning About Natural Language Grounded in Photographs
We introduce a new dataset for joint reasoning about language and vision...
research
∙
09/04/2018
Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction
We propose to decompose instruction execution to goal prediction and act...
research
∙
05/31/2018
Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning
We introduce a method for following high-level navigation instructions b...
research
∙
05/25/2018
Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation
We propose a learning approach for mapping context-dependent sequential ...
research
∙
04/30/2018
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies
We present NEWSROOM, a summarization dataset of 1.3 million articles and...
research
∙
04/18/2018
Learning to Map Context-Dependent Sentences to Executable Formal Queries
We propose a context-dependent model to map utterances within an interac...
research
∙
01/23/2018
CHALET: Cornell House Agent Learning Environment
We present CHALET, a 3D house simulator with support for navigation and ...
research
∙
10/02/2017
Visual Reasoning with Natural Language
Natural language provides a widely accessible and expressive interface f...
research
∙
09/08/2017
Training RNNs as Fast as CNNs
Common recurrent neural network architectures scale poorly due to the in...
research
∙
04/28/2017