Ruslan Salakhutdinov

research

∙ 06/30/2022

MultiViz: An Analysis Benchmark for Visualizing and Understanding Multimodal Models

The promise of multimodal models for real-world applications has inspire...

Paul Pu Liang, et al. ∙

research

∙ 06/21/2022

A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search

Physically rearranging objects is an important capability for embodied a...

Brandon Trabucco, et al. ∙

research

∙ 06/15/2022

Contrastive Learning as Goal-Conditioned Reinforcement Learning

In reinforcement learning (RL), it is easier to solve a task if given a ...

Benjamin Eysenbach, et al. ∙

research

∙ 06/07/2022

Imitating Past Successes can be Very Suboptimal

Prior work has proposed a simple strategy for reinforcement learning (RL...

Benjamin Eysenbach, et al. ∙

research

∙ 05/25/2022

Reasoning over Logically Interacted Conditions for Question Answering

Some questions have multiple answers that are not equally correct, i.e. ...

Haitian Sun, et al. ∙

research

∙ 03/21/2022

PACS: A Dataset for Physical Audiovisual CommonSense Reasoning

In order for AI to be safely deployed in real-world scenarios such as ho...

Samuel Yu, et al. ∙

research

∙ 03/03/2022

Zero-shot Domain Adaptation of Heterogeneous Graphs via Knowledge Transfer Networks

How can we make predictions for nodes in a heterogeneous graph when an e...

Minji Yoon, et al. ∙

research

∙ 03/03/2022

DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations

The ability for a human to understand an Artificial Intelligence (AI) mo...

Yiwei Lyu, et al. ∙

research

∙ 03/02/2022

HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning

Learning multimodal representations involves discovering correspondences...

Paul Pu Liang, et al. ∙

research

∙ 02/14/2022

Learning Weakly-Supervised Contrastive Representations

We argue that a form of the valuable information provided by the auxilia...

Yao-Hung Hubert Tsai, et al. ∙

research

∙ 02/11/2022

Conditional Contrastive Learning with Kernel

Conditional contrastive learning frameworks consider the conditional sam...

Yao-Hung Hubert Tsai, et al. ∙

research

∙ 12/02/2021

SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency

In this paper, we explore how we can build upon the data and models of I...

Devendra Singh Chaplot, et al. ∙

research

∙ 10/28/2021

Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives

Despite the potential of reinforcement learning (RL) for building genera...

Murtaza Dalal, et al. ∙

research

∙ 10/22/2021

C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks

Goal-conditioned reinforcement learning (RL) can solve tasks in a wide r...

Tianjun Zhang, et al. ∙

research

∙ 10/13/2021

ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers

We describe a Question Answering (QA) dataset that contains complex ques...

Haitian Sun, et al. ∙

research

∙ 10/12/2021

FILM: Following Instructions in Language with Modular Methods

Recent methods for embodied instruction following are typically trained ...

So Yeon Min, et al. ∙

research

∙ 10/11/2021

Recurrent Model-Free RL is a Strong Baseline for Many POMDPs

Many problems in RL, such as meta RL, robust RL, and generalization in R...

Tianwei Ni, et al. ∙

research

∙ 10/06/2021

Mismatched No More: Joint Model-Policy Optimization for Model-Based RL

Many model-based reinforcement learning (RL) methods follow a similar te...

Benjamin Eysenbach, et al. ∙

research

∙ 10/06/2021

The Information Geometry of Unsupervised Reinforcement Learning

How can a reinforcement learning (RL) agent prepare to solve downstream ...

Benjamin Eysenbach, et al. ∙

research

∙ 09/27/2021

FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding

The few-shot natural language understanding (NLU) task has attracted muc...

Yanan Zheng, et al. ∙

research

∙ 09/07/2021

Robust Predictable Control

Many of the challenges facing today's reinforcement learning (RL) algori...

Benjamin Eysenbach, et al. ∙

research

∙ 07/15/2021

MultiBench: Multiscale Benchmarks for Multimodal Representation Learning

Learning multimodal representations involves integrating information fro...

Paul Pu Liang, et al. ∙

research

∙ 06/24/2021

Towards Understanding and Mitigating Social Biases in Language Models

As machine learning methods are deployed in real-world settings such as ...

Paul Pu Liang, et al. ∙

research

∙ 06/24/2021

Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data

Mental health conditions remain underdiagnosed even in countries with co...

Paul Pu Liang, et al. ∙

research

∙ 06/14/2021

HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

Self-supervised approaches for speech representation learning are challe...

Wei-Ning Hsu, et al. ∙

research

∙ 06/14/2021

Online Sub-Sampling for Reinforcement Learning with General Function Approximation

Designing provably efficient algorithms with general function approximat...

Dingwen Kong, et al. ∙

research

∙ 06/05/2021

Integrating Auxiliary Information in Self-supervised Learning

This paper presents to integrate the auxiliary information (e.g., additi...

Yao-Hung Hubert Tsai, et al. ∙

research

∙ 06/05/2021

Conditional Contrastive Learning: Removing Undesirable Information in Self-Supervised Representations

Self-supervised learning is a form of unsupervised learning that leverag...

Yao-Hung Hubert Tsai, et al. ∙

research

∙ 06/01/2021

End-to-End Multihop Retrieval for Compositional Question Answering over Long Documents

Answering complex questions from long documents requires aggregating mul...

Haitian Sun, et al. ∙

research

∙ 05/17/2021

Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning

Offline Reinforcement Learning promises to learn effective policies from...

Yue Wu, et al. ∙

research

∙ 04/28/2021

A Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive Learning

In this report, we relate the algorithmic design of Barlow Twins' method...

Yao-Hung Hubert Tsai, et al. ∙

research

∙ 04/26/2021

Focused Attention Improves Document-Grounded Generation

Document grounded generation is the task of using the information provid...

Shrimai Prabhumoye, et al. ∙

research

∙ 04/04/2021

Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation

Many real-world applications such as robotics provide hard constraints o...

Emilio Parisotto, et al. ∙

research

∙ 03/23/2021

Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classification

In the standard Markov decision process formalism, users specify tasks b...

Benjamin Eysenbach, et al. ∙

research

∙ 03/21/2021

Self-supervised Representation Learning with Relative Predictive Coding

This paper introduces Relative Predictive Coding (RPC), a new contrastiv...

Yao-Hung Hubert Tsai, et al. ∙

research

∙ 03/08/2021

Instabilities of Offline RL with Pre-Trained Neural Representation

In offline reinforcement learning (RL), we seek to utilize offline data ...

Ruosong Wang, et al. ∙

research

∙ 02/20/2021

On Proximal Policy Optimization's Heavy-tailed Gradients

Modern policy gradient algorithms, notably Proximal Policy Optimization ...

Saurabh Garg, et al. ∙

research

∙ 02/14/2021

Reasoning Over Virtual Knowledge Bases With Open Predicate Relations

We present the Open Predicate Query Language (OPQL); a method for constr...

Haitian Sun, et al. ∙

research

∙ 01/26/2021

The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors

Although deep reinforcement learning has led to breakthroughs in many di...

William H. Guss, et al. ∙

research

∙ 01/22/2021

Understanding the Tradeoffs in Client-Side Privacy for Speech Recognition

Existing approaches to ensuring privacy of user speech data primarily fo...

Peter Wu, et al. ∙

research

∙ 12/04/2020

Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment

The natural world is abundant with concepts expressed via visual, acoust...

Paul Pu Liang, et al. ∙

research

∙ 11/17/2020

C-Learning: Learning to Achieve Goals via Recursive Classification

We study the problem of predicting and controlling the future state dist...

Benjamin Eysenbach, et al. ∙

research

∙ 11/17/2020

Close Category Generalization

Out-of-distribution generalization is a core challenge in machine learni...

Yao-Yuan Yang, et al. ∙

research

∙ 10/27/2020

Unsupervised Domain Adaptation for Visual Navigation

Advances in visual navigation methods have led to intelligent embodied n...

Shangda Li, et al. ∙

research

∙ 10/22/2020

Planning with Submodular Objective Functions

We study planning with submodular objective functions, where instead of ...

Ruosong Wang, et al. ∙

research

∙ 10/09/2020

Case Study: Deontological Ethics in NLP

Recent work in natural language processing (NLP) has focused on ethical ...

Shrimai Prabhumoye, et al. ∙

research

∙ 09/28/2020

Graph Adversarial Networks: Protecting Information against Adversarial Attacks

We study the problem of protecting information when learning with graph ...

Peiyuan Liao, et al. ∙

research

∙ 09/08/2020

Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function

In this paper, we study bidirectional LSTM network for the task of text ...

Devendra Singh Sachan, et al. ∙

research

∙ 08/22/2020

Few-Shot Learning with Intra-Class Knowledge Transfer

We consider the few-shot classification task with an unbalanced dataset,...

Vivek Roy, et al. ∙

research

∙ 07/16/2020

Towards Debiasing Sentence Representations

As natural language processing methods are increasingly deployed in real...

Paul Pu Liang, et al. ∙

Ruslan Salakhutdinov

Featured Co-authors