b'Yonatan Bisk'

research

∙ 09/18/2023

Reasoning about the Unseen for Efficient Outdoor Object Navigation

Robots should exist anywhere humans do: indoors, outdoors, and even unma...

0 Quanting Xie, et al. ∙

research

∙ 09/15/2023

MOSAIC: Learning Unified Multi-Sensory Object Property Representations for Robot Perception

A holistic understanding of object properties across diverse sensory mod...

0 Gyan Tatiya, et al. ∙

research

∙ 07/25/2023

MAEA: Multimodal Attribution for Embodied AI

Understanding multimodal perception for embodied AI is an open question ...

0 Vidhi Jain, et al. ∙

research

∙ 06/20/2023

HomeRobot: Open-Vocabulary Mobile Manipulation

HomeRobot (noun): An affordable compliant robot that navigates homes and...

3 Sriram Yenamandra, et al. ∙

research

∙ 05/24/2023

SPRING: GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning

Open-world survival games pose significant challenges for AI algorithms ...

0 Yue Wu, et al. ∙

research

∙ 05/03/2023

Plan, Eliminate, and Track – Language Models are Good Teachers for Embodied Agents

Pre-trained large language models (LLMs) capture procedural knowledge ab...

0 Yue Wu, et al. ∙

research

∙ 02/13/2023

The Framework Tax: Disparities Between Inference Efficiency in Research and Deployment

Increased focus on the deployment of machine learning systems has led to...

0 Jared Fernandez, et al. ∙

research

∙ 12/09/2022

Object Goal Navigation with End-to-End Self-Supervision

A household robot should be able to navigate to target locations without...

0 So Yeon Min, et al. ∙

research

∙ 07/06/2022

Transformers are Adaptable Task Planners

Every home is different, and every person likes things done in their par...

27 Vidhi Jain, et al. ∙

research

∙ 05/24/2022

On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization

Integrating vision and language has gained notable attention following t...

0 Shruti Palaskar, et al. ∙

research

∙ 05/19/2022

Training Vision-Language Transformers from Captions Alone

We show that Vision-Language Transformers can be learned without human l...

0 Liangke Gui, et al. ∙

research

∙ 03/06/2022

HEAR 2021: Holistic Evaluation of Audio Representations

What audio embedding approach generalizes best to a wide range of downst...

17 Joseph Turian, et al. ∙

research

∙ 12/17/2021

It's Time to Do Something: Mitigating the Negative Impacts of Computing Through a Change to the Peer Review Process

The computing research community needs to work much harder to address th...

0 Brent Hecht, et al. ∙

research

∙ 12/16/2021

KAT: A Knowledge Augmented Transformer for Vision-and-Language

The primary focus of recent work with largescale transformers has been o...

0 Liangke Gui, et al. ∙

research

∙ 10/14/2021

Learning When and What to Ask: a Hierarchical Reinforcement Learning Framework

Reliable AI agents should be mindful of the limits of their knowledge an...

0 Khanh Nguyen, et al. ∙

research

∙ 10/12/2021

FILM: Following Instructions in Language with Modular Methods

Recent methods for embodied instruction following are typically trained ...

1 So Yeon Min, et al. ∙

research

∙ 09/01/2021

WebQA: Multihop and Multimodal QA

Web search is fundamentally multimodal and multihop. Often, even before ...

0 Yingshan Chang, et al. ∙

research

∙ 08/23/2021

TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment

Contrastive learning has been widely used to train transformer-based vis...

2 Jianwei Yang, et al. ∙

research

∙ 07/26/2021

Language Grounding with 3D Objects

Seemingly simple natural language requests to a robot are generally unde...

4 Jesse Thomason, et al. ∙

research

∙ 06/04/2021

Grounding 'Grounding' in NLP

The NLP community has seen substantial recent interest in grounding to f...

10 Khyathi Raghavi Chandu, et al. ∙

research

∙ 04/18/2021

Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Models

Numerous works have analyzed biases in vision and pre-trained language m...

13 Tejas Srinivasan, et al. ∙

research

∙ 01/31/2021

An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games

Guessing games are a prototypical instance of the "learning by interacti...

0 Alessandro Suglia, et al. ∙

research

∙ 11/07/2020

Knowledge-driven Self-supervision for Zero-shot Commonsense Question Answering

Recent developments in pre-trained neural language modeling have led to ...

9 Kaixin Ma, et al. ∙

research

∙ 11/05/2020

Imagining Grounded Conceptual Representations from Perceptual Information in Situated Guessing Games

In visual guessing games, a Guesser has to identify a target object in a...

0 Alessandro Suglia, et al. ∙

research

∙ 10/08/2020

ALFWorld: Aligning Text and Embodied Environments for Interactive Learning

Given a simple request (e.g., Put a washed apple in the kitchen fridge),...

10 Mohit Shridhar, et al. ∙

research

∙ 07/29/2020

The Return of Lexical Dependencies: Neural Lexicalized PCFGs

In this paper we demonstrate that context free grammar (CFG) based metho...

0 Yonatan Bisk, et al. ∙

research

∙ 05/02/2020

RMM: A Recursive Mental Model for Dialog Navigation

Fluent communication requires understanding your audience. In the new co...

0 Homero Roman Roman, et al. ∙

research

∙ 05/02/2020

A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos

Procedural knowledge, which we define as concrete information about the ...

6 Frank F. Xu, et al. ∙

research

∙ 04/21/2020

Experience Grounds Language

Successful linguistic communication relies on a shared experience of the...

1 Yonatan Bisk, et al. ∙

research

∙ 03/02/2020

Multi-View Learning for Vision-and-Language Navigation

Learning to navigate in a visual environment following natural language ...

7 Qiaolin Xia, et al. ∙

research

∙ 12/03/2019

ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

We present ALFRED (Action Learning From Realistic Environments and Direc...

5 Mohit Shridhar, et al. ∙

research

∙ 11/26/2019

PIQA: Reasoning about Physical Commonsense in Natural Language

To apply eyeshadow without a brush, should I use a cotton swab or a toot...

51 Yonatan Bisk, et al. ∙

research

∙ 09/05/2019

Robust Navigation with Language Pretraining and Stochastic Sampling

Core to the vision-and-language navigation (VLN) challenge is building r...

5 Xiujun Li, et al. ∙

research

∙ 05/29/2019

Defending Against Neural Fake News

Recent progress in natural language generation has raised dual-use conce...

0 Rowan Zellers, et al. ∙

research

∙ 05/19/2019

HellaSwag: Can a Machine Really Finish Your Sentence?

Recent work by Zellers et al. (2018) introduced a new task of commonsens...

0 Rowan Zellers, et al. ∙

research

∙ 04/02/2019

Improving Robot Success Detection using Static Object Data

We use static object data to improve success detection for stacking obje...

0 Rosario Scalise, et al. ∙

research

∙ 03/20/2019

Prospection: Interpretable Plans From Language By Predicting the Future

High-level human instructions often correspond to behaviors with multipl...

8 Chris Paxton, et al. ∙

research

∙ 03/06/2019

Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation

We present FAST NAVIGATOR, a general framework for action decoding, whic...

6 Liyiming Ke, et al. ∙

research

∙ 02/02/2019

Character-based Surprisal as a Model of Human Reading in the Presence of Errors

Intuitively, human readers cope easily with errors in text; typos, missp...

0 Michael Hahn, et al. ∙

research

∙ 11/27/2018

From Recognition to Cognition: Visual Commonsense Reasoning

Visual understanding goes well beyond object recognition. With one glanc...

30 Rowan Zellers, et al. ∙

research

∙ 11/21/2018

Early Fusion for Goal Directed Robotic Vision

Increasingly, perceptual systems are being codified as strict pipelines ...

10 Aaron Walsman, et al. ∙

research

∙ 11/01/2018

Shifting the Baseline: Single Modality Performance on Visual Navigation & QA

Language-and-vision navigation and question answering (QA) are exciting ...

0 Jesse Thomason, et al. ∙

research

∙ 08/16/2018

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference

Given a partial description like "she opened the hood of the car," human...

0 Rowan Zellers, et al. ∙

research

∙ 05/28/2018

Inducing Grammars with and for Neural Machine Translation

Machine translation systems require semantic knowledge and grammatical u...

0 Ke Tran, et al. ∙

research

∙ 05/20/2018

Balancing Shared Autonomy with Human-Robot Communication

Robotic agents that share autonomy with a human should leverage human do...

0 Rosario Scalise, et al. ∙

research

∙ 01/23/2018

CHALET: Cornell House Agent Learning Environment

We present CHALET, a 3D house simulator with support for navigation and ...

0 Claudia Yan, et al. ∙

research

∙ 12/10/2017

Learning Interpretable Spatial Operations in a Rich 3D Blocks World

In this paper, we study the problem of mapping natural language instruct...

0 Yonatan Bisk, et al. ∙

research

∙ 11/06/2017

Synthetic and Natural Noise Both Break Neural Machine Translation

Character-based neural machine translation (NMT) models alleviate out-of...

0 Yonatan Belinkov, et al. ∙

research

∙ 10/09/2017

Natural Language Inference from Multiple Premises

We define a novel textual entailment task that requires inference over m...

0 Alice Lai, et al. ∙

research

∙ 09/29/2016

Evaluating Induced CCG Parsers on Grounded Semantic Parsing

We compare the effectiveness of four different syntactic CCG parsers for...

0 Yonatan Bisk, et al. ∙

Yonatan Bisk

Featured Co-authors