b'Yonatan Bisk'

Language Grounding with 3D Objects

Seemingly simple natural language requests to a robot are generally unde...
07/26/2021 ∙ by Jesse Thomason, et al. ∙ 3 ∙ share

read it
Grounding 'Grounding' in NLP

The NLP community has seen substantial recent interest in grounding to f...
06/04/2021 ∙ by Khyathi Raghavi Chandu, et al. ∙ 10 ∙ share

read it
Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Models

Numerous works have analyzed biases in vision and pre-trained language m...
04/18/2021 ∙ by Tejas Srinivasan, et al. ∙ 13 ∙ share

read it
An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games

Guessing games are a prototypical instance of the "learning by interacti...
01/31/2021 ∙ by Alessandro Suglia, et al. ∙ 0 ∙ share

read it
Knowledge-driven Self-supervision for Zero-shot Commonsense Question Answering

Recent developments in pre-trained neural language modeling have led to ...
11/07/2020 ∙ by Kaixin Ma, et al. ∙ 9 ∙ share

read it
Imagining Grounded Conceptual Representations from Perceptual Information in Situated Guessing Games

In visual guessing games, a Guesser has to identify a target object in a...
11/05/2020 ∙ by Alessandro Suglia, et al. ∙ 0 ∙ share

read it
ALFWorld: Aligning Text and Embodied Environments for Interactive Learning

Given a simple request (e.g., Put a washed apple in the kitchen fridge),...
10/08/2020 ∙ by Mohit Shridhar, et al. ∙ 10 ∙ share

read it
The Return of Lexical Dependencies: Neural Lexicalized PCFGs

In this paper we demonstrate that context free grammar (CFG) based metho...
07/29/2020 ∙ by Yonatan Bisk, et al. ∙ 0 ∙ share

read it
RMM: A Recursive Mental Model for Dialog Navigation

Fluent communication requires understanding your audience. In the new co...
05/02/2020 ∙ by Homero Roman Roman, et al. ∙ 0 ∙ share

read it
A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos

Procedural knowledge, which we define as concrete information about the ...
05/02/2020 ∙ by Frank F. Xu, et al. ∙ 6 ∙ share

read it
Experience Grounds Language

Successful linguistic communication relies on a shared experience of the...
04/21/2020 ∙ by Yonatan Bisk, et al. ∙ 1 ∙ share

read it
Multi-View Learning for Vision-and-Language Navigation

Learning to navigate in a visual environment following natural language ...
03/02/2020 ∙ by Qiaolin Xia, et al. ∙ 7 ∙ share

read it
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

We present ALFRED (Action Learning From Realistic Environments and Direc...
12/03/2019 ∙ by Mohit Shridhar, et al. ∙ 5 ∙ share

read it
PIQA: Reasoning about Physical Commonsense in Natural Language

To apply eyeshadow without a brush, should I use a cotton swab or a toot...
11/26/2019 ∙ by Yonatan Bisk, et al. ∙ 51 ∙ share

read it
Robust Navigation with Language Pretraining and Stochastic Sampling

Core to the vision-and-language navigation (VLN) challenge is building r...
09/05/2019 ∙ by Xiujun Li, et al. ∙ 5 ∙ share

read it
Defending Against Neural Fake News

Recent progress in natural language generation has raised dual-use conce...
05/29/2019 ∙ by Rowan Zellers, et al. ∙ 0 ∙ share

read it
HellaSwag: Can a Machine Really Finish Your Sentence?

Recent work by Zellers et al. (2018) introduced a new task of commonsens...
05/19/2019 ∙ by Rowan Zellers, et al. ∙ 0 ∙ share

read it
Improving Robot Success Detection using Static Object Data

We use static object data to improve success detection for stacking obje...
04/02/2019 ∙ by Rosario Scalise, et al. ∙ 0 ∙ share

read it
Prospection: Interpretable Plans From Language By Predicting the Future

High-level human instructions often correspond to behaviors with multipl...
03/20/2019 ∙ by Chris Paxton, et al. ∙ 8 ∙ share

read it
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation

We present FAST NAVIGATOR, a general framework for action decoding, whic...
03/06/2019 ∙ by Liyiming Ke, et al. ∙ 6 ∙ share

read it
Character-based Surprisal as a Model of Human Reading in the Presence of Errors

Intuitively, human readers cope easily with errors in text; typos, missp...
02/02/2019 ∙ by Michael Hahn, et al. ∙ 0 ∙ share

read it
From Recognition to Cognition: Visual Commonsense Reasoning

Visual understanding goes well beyond object recognition. With one glanc...
11/27/2018 ∙ by Rowan Zellers, et al. ∙ 30 ∙ share

read it
Early Fusion for Goal Directed Robotic Vision

Increasingly, perceptual systems are being codified as strict pipelines ...
11/21/2018 ∙ by Aaron Walsman, et al. ∙ 10 ∙ share

read it
Shifting the Baseline: Single Modality Performance on Visual Navigation & QA

Language-and-vision navigation and question answering (QA) are exciting ...
11/01/2018 ∙ by Jesse Thomason, et al. ∙ 0 ∙ share

read it
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference

Given a partial description like "she opened the hood of the car," human...
08/16/2018 ∙ by Rowan Zellers, et al. ∙ 0 ∙ share

read it
Inducing Grammars with and for Neural Machine Translation

Machine translation systems require semantic knowledge and grammatical u...
05/28/2018 ∙ by Ke Tran, et al. ∙ 0 ∙ share

read it
Balancing Shared Autonomy with Human-Robot Communication

Robotic agents that share autonomy with a human should leverage human do...
05/20/2018 ∙ by Rosario Scalise, et al. ∙ 0 ∙ share

read it
CHALET: Cornell House Agent Learning Environment

We present CHALET, a 3D house simulator with support for navigation and ...
01/23/2018 ∙ by Claudia Yan, et al. ∙ 0 ∙ share

read it
Learning Interpretable Spatial Operations in a Rich 3D Blocks World

In this paper, we study the problem of mapping natural language instruct...
12/10/2017 ∙ by Yonatan Bisk, et al. ∙ 0 ∙ share

read it
Synthetic and Natural Noise Both Break Neural Machine Translation

Character-based neural machine translation (NMT) models alleviate out-of...
11/06/2017 ∙ by Yonatan Belinkov, et al. ∙ 0 ∙ share

read it
Natural Language Inference from Multiple Premises

We define a novel textual entailment task that requires inference over m...
10/09/2017 ∙ by Alice Lai, et al. ∙ 0 ∙ share

read it
Evaluating Induced CCG Parsers on Grounded Semantic Parsing

We compare the effectiveness of four different syntactic CCG parsers for...
09/29/2016 ∙ by Yonatan Bisk, et al. ∙ 0 ∙ share

read it

Yonatan Bisk

Featured Co-authors