b'Yoav Artzi'

research

∙ 09/06/2023

A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models

Key to tasks that require reasoning about natural language in visual con...

0 Noriyuki Kojima, et al. ∙

research

∙ 07/19/2023

IncDSI: Incrementally Updatable Document Retrieval

Differentiable Search Index is a recently proposed paradigm for document...

0 Varsha Kishore, et al. ∙

research

∙ 05/21/2023

Continually Improving Extractive QA via Human Feedback

We study continually improving an extractive question answering (QA) sys...

0 Ge Gao, et al. ∙

research

∙ 05/11/2023

Semantic uncertainty guides the extension of conventions to new referents

A long tradition of studies in psycholinguistics has examined the format...

0 Ron Eliav, et al. ∙

research

∙ 03/14/2023

CB2: Collaborative Natural Language Interaction Research Platform

CB2 is a multi-agent platform to study collaborative natural language in...

0 Jacob Sharf, et al. ∙

research

∙ 12/19/2022

Continual Learning for Instruction Following from Realtime Feedback

We study the problem of continually training an instruction-following ag...

0 Alane Suhr, et al. ∙

research

∙ 11/29/2022

Abstract Visual Reasoning with Tangram Shapes

We introduce KiloGram, a resource for studying abstract visual reasoning...

0 Anya Ji, et al. ∙

research

∙ 11/03/2022

lilGym: Natural Language Visual Reasoning with Reinforcement Learning

We present lilGym, a new benchmark for language-conditioned reinforcemen...

14 Anne Wu, et al. ∙

research

∙ 05/02/2022

Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages

We introduce Wav2Seq, the first self-supervised approach to pre-train bo...

0 Felix Wu, et al. ∙

research

∙ 03/18/2022

Simulating Bandit Learning from User Feedback for Extractive Question Answering

We study learning from user feedback for extractive question answering b...

0 Ge Gao, et al. ∙

research

∙ 11/19/2021

SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech

Progress in speech processing has been facilitated by shared datasets an...

21 Suwon Shon, et al. ∙

research

∙ 09/28/2021

When in Doubt: Improving Classification Performance with Alternating Normalization

We introduce Classification with Alternating Normalization (CAN), a non-...

7 Menglin Jia, et al. ∙

research

∙ 09/14/2021

Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition

This paper is a study of performance-efficiency trade-offs in pre-traine...

0 Felix Wu, et al. ∙

research

∙ 09/09/2021

Analysis of Language Change in Collaborative Instruction Following

We analyze language change over time in a collaborative, goal-oriented i...

0 Anna Effenberger, et al. ∙

research

∙ 08/16/2021

Who's Waldo? Linking People Across Text and Images

We present a task and benchmark dataset for person-centric visual ground...

0 Claire Yuqing Cui, et al. ∙

research

∙ 08/10/2021

Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior

We study continual learning for natural language instruction generation,...

0 Noriyuki Kojima, et al. ∙

research

∙ 07/12/2021

A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution

Natural language provides an accessible and expressive interface to spec...

18 Valts Blukis, et al. ∙

research

∙ 11/14/2020

Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following

We study the problem of learning a robot policy to follow natural langua...

1 Valts Blukis, et al. ∙

research

∙ 06/10/2020

Revisiting Few-sample BERT Fine-tuning

We study the problem of few-sample fine-tuning of BERT contextual repres...

0 Tianyi Zhang, et al. ∙

research

∙ 05/04/2020

What is Learned in Visually Grounded Neural Syntax Acquisition

Visual features are a promising signal for learning bootstrap textual mo...

0 Noriyuki Kojima, et al. ∙

research

∙ 04/06/2020

Evaluating NLP Models via Contrast Sets

Standard test sets for supervised learning evaluate in-distribution gene...

0 Matt Gardner, et al. ∙

research

∙ 01/10/2020

Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View

The Touchdown dataset (Chen et al., 2019) provides instructions by human...

9 Harsh Mehta, et al. ∙

research

∙ 11/09/2019

Interactive Classification by Asking Informative Questions

Natural language systems often rely on a single, potentially ambiguous i...

0 Lili Yu, et al. ∙

research

∙ 10/21/2019

Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight

We propose a joint simulation and real-world learning framework for mapp...

21 Valts Blukis, et al. ∙

research

∙ 10/08/2019

Executing Instructions in Situated Collaborative Interactions

We study a collaborative scenario where a user not only instructs a syst...

11 Alane Suhr, et al. ∙

research

∙ 09/23/2019

NLVR2 Visual Bias Analysis

NLVR2 (Suhr et al., 2019) was designed to be robust for language bias th...

0 Alane Suhr, et al. ∙

research

∙ 04/21/2019

BERTScore: Evaluating Text Generation with BERT

We propose BERTScore, an automatic evaluation metric for text generation...

0 Tianyi Zhang, et al. ∙

research

∙ 11/29/2018

Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments

We study the problem of jointly reasoning about language and vision thro...

10 Howard Chen, et al. ∙

research

∙ 11/21/2018

Early Fusion for Goal Directed Robotic Vision

Increasingly, perceptual systems are being codified as strict pipelines ...

10 Aaron Walsman, et al. ∙

research

∙ 11/10/2018

Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction

We propose an approach for mapping natural language instructions and raw...

0 Valts Blukis, et al. ∙

research

∙ 11/01/2018

A Corpus for Reasoning About Natural Language Grounded in Photographs

We introduce a new dataset for joint reasoning about language and vision...

2 Alane Suhr, et al. ∙

research

∙ 09/04/2018

Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction

We propose to decompose instruction execution to goal prediction and act...

0 Dipendra Misra, et al. ∙

research

∙ 05/31/2018

Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning

We introduce a method for following high-level navigation instructions b...

0 Valts Blukis, et al. ∙

research

∙ 05/25/2018

Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation

We propose a learning approach for mapping context-dependent sequential ...

0 Alane Suhr, et al. ∙

research

∙ 04/30/2018

Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies

We present NEWSROOM, a summarization dataset of 1.3 million articles and...

0 Max Grusky, et al. ∙

research

∙ 04/18/2018

Learning to Map Context-Dependent Sentences to Executable Formal Queries

We propose a context-dependent model to map utterances within an interac...

0 Alane Suhr, et al. ∙

research

∙ 01/23/2018

CHALET: Cornell House Agent Learning Environment

We present CHALET, a 3D house simulator with support for navigation and ...

0 Claudia Yan, et al. ∙

research

∙ 10/02/2017

Visual Reasoning with Natural Language

Natural language provides a widely accessible and expressive interface f...

0 Stephanie Zhou, et al. ∙

research

∙ 09/08/2017

Training RNNs as Fast as CNNs

Common recurrent neural network architectures scale poorly due to the in...

0 Tao Lei, et al. ∙

research

∙ 04/28/2017

Mapping Instructions and Visual Observations to Actions with Reinforcement Learning

We propose to directly map raw visual observations and text input to act...

0 Dipendra Misra, et al. ∙

Yoav Artzi

Featured Co-authors