b'Yuandong Tian'

research

∙ 07/24/2023

RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment

We propose Reinforcement Learning from Contrast Distillation (RLCD), a m...

0 Kevin Yang, et al. ∙

research

∙ 07/18/2023

Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information

Recent works in learning-integrated optimization have shown promise in s...

0 Arman Zharmagambetov, et al. ∙

research

∙ 06/27/2023

Extending Context Window of Large Language Models via Positional Interpolation

We present Position Interpolation (PI) that extends the context window s...

0 Shouyuan Chen, et al. ∙

research

∙ 06/24/2023

H_2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models

Large Language Models (LLMs), despite their recent impressive accomplish...

0 Zhenyu Zhang, et al. ∙

research

∙ 05/25/2023

Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer

Transformer architecture has shown impressive performance in multiple re...

20 Yuandong Tian, et al. ∙

research

∙ 05/03/2023

Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models

Sharding a large machine learning model across multiple devices to balan...

4 Daochen Zha, et al. ∙

research

∙ 04/24/2023

A Cookbook of Self-Supervised Learning

Self-supervised learning, dubbed the dark matter of intelligence, is a p...

0 Randall Balestriero, et al. ∙

research

∙ 02/03/2023

Searching Large Neighborhoods for Integer Linear Programs with Contrastive Learning

Integer Linear Programs (ILPs) are powerful tools for modeling and solvi...

0 Taoan Huang, et al. ∙

research

∙ 01/09/2023

Learning to compile smartly for program size reduction

Compiler optimization passes are an important tool for improving program...

1 Youwei Liang, et al. ∙

research

∙ 12/20/2022

DOC: Improving Long Story Coherence With Detailed Outline Control

We propose the Detailed Outline Control (DOC) framework for improving lo...

5 Kevin Yang, et al. ∙

research

∙ 12/15/2022

Local Branching Relaxation Heuristics for Integer Linear Programs

Large Neighborhood Search (LNS) is a popular heuristic algorithm for sol...

13 Taoan Huang, et al. ∙

research

∙ 11/23/2022

EurNet: Efficient Multi-Range Relational Modeling of Spatial Multi-Relational Data

Modeling spatial relationship in the data remains critical across many d...

4 Minghao Xu, et al. ∙

research

∙ 10/22/2022

SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problems

Optimization problems with expensive nonlinear cost functions and combin...

7 Aaron Ferber, et al. ∙

research

∙ 10/13/2022

Re3: Generating Longer Stories With Recursive Reprompting and Revision

We consider the problem of automatically generating longer stories of ov...

4 Kevin Yang, et al. ∙

research

∙ 08/22/2022

Efficient Planning in a Compact Latent Action Space

While planning-based sequence modelling methods have shown great potenti...

4 Zhengyao Jiang, et al. ∙

research

∙ 08/12/2022

AutoShard: Automated Embedding Table Sharding for Recommender Systems

Embedding learning is an important technique in deep recommendation mode...

9 Daochen Zha, et al. ∙

research

∙ 06/30/2022

Denoised MDPs: Learning World Models Better Than the World Itself

The ability to separate signal from noise, and reason with clean abstrac...

8 Tongzhou Wang, et al. ∙

research

∙ 06/02/2022

Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning

While the empirical success of self-supervised learning (SSL) heavily re...

13 Yuandong Tian, et al. ∙

research

∙ 04/01/2022

On the Importance of Asymmetry for Siamese Representation Learning

Many recent self-supervised frameworks for visual representation learnin...

5 Xiao Wang, et al. ∙

research

∙ 02/11/2022

Understanding Curriculum Learning in Policy Optimization for Solving Combinatorial Optimization Problems

Over the recent years, reinforcement learning (RL) has shown impressive ...

9 Runlong Zhou, et al. ∙

research

∙ 01/29/2022

Deep Contrastive Learning is Provably (almost) Principal Component Analysis

We show that Contrastive Learning (CL) under a family of loss functions ...

31 Yuandong Tian, et al. ∙

research

∙ 12/16/2021

Learning Bounded Context-Free-Grammar via LSTM and the Transformer:Difference and Explanations

Long Short-Term Memory (LSTM) and Transformers are two popular neural ar...

15 Hui Shi, et al. ∙

research

∙ 11/19/2021

Dynamic Graph Representation Learning via Graph Transformer Networks

Dynamic graph representation learning is an important task with widespre...

10 Weilin Cong, et al. ∙

research

∙ 10/11/2021

Towards Demystifying Representation Learning with Non-contrastive Self-supervision

Non-contrastive methods of self-supervised learning (such as BYOL and Si...

2 Xiang Wang, et al. ∙

research

∙ 10/07/2021

Multi-objective Optimization by Learning Space Partitions

In contrast to single-objective optimization (SOO), multi-objective opti...

18 Yiyang Zhao, et al. ∙

research

∙ 09/17/2021

CompilerGym: Robust, Performant Compiler Optimization Environments for AI Research

Interest in applying Artificial Intelligence (AI) techniques to compiler...

17 Chris Cummins, et al. ∙

research

∙ 06/29/2021

Latent Execution for Neural Program Synthesis Beyond Domain-Specific Languages

Program synthesis from input-output examples has been a long-standing ch...

3 Xinyun Chen, et al. ∙

research

∙ 06/19/2021

Learning Space Partitions for Path Planning

Path planning, the problem of efficiently discovering high-reward trajec...

5 Kevin Yang, et al. ∙

research

∙ 06/18/2021

MADE: Exploration via Maximizing Deviation from Explored Regions

In online reinforcement learning (RL), efficient exploration remains par...

4 Tianjun Zhang, et al. ∙

research

∙ 02/25/2021

Understanding Robustness in Teacher-Student Setting: A New Perspective

Adversarial examples have appeared as a ubiquitous property of machine l...

13 Zhuolin Yang, et al. ∙

research

∙ 02/17/2021

Provably Efficient Policy Gradient Methods for Two-Player Zero-Sum Markov Games

Policy gradient methods are widely used in solving two-player zero-sum g...

11 Yulai Zhao, et al. ∙

research

∙ 02/12/2021

Understanding self-supervised Learning Dynamics without Contrastive Pairs

Contrastive approaches to self-supervised learning (SSL) learn represent...

9 Yuandong Tian, et al. ∙

research

∙ 11/22/2020

FP-NAS: Fast Probabilistic Neural Architecture Search

Differential Neural Architecture Search (NAS) requires all layer choices...

4 Zhicheng Yan, et al. ∙

research

∙ 10/01/2020

Understanding Self-supervised Learning with Dual Deep Networks

We propose a novel theoretical framework to understand self-supervised l...

52 Yuandong Tian, et al. ∙

research

∙ 08/28/2020

Real-world Video Adaptation with Reinforcement Learning

Client-side video players employ adaptive bitrate (ABR) algorithms to op...

8 Hongzi Mao, et al. ∙

research

∙ 08/27/2020

Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion

Hierarchical learning has been successful at learning generalizable loco...

0 Tianyu Li, et al. ∙

research

∙ 08/14/2020

Joint Policy Search for Multi-agent Collaboration with Imperfect Information

To learn good joint policies for multi-agent collaboration with imperfec...

13 Yuandong Tian, et al. ∙

research

∙ 07/01/2020

Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search

High dimensional black-box optimization has broad applications but remai...

23 Linnan Wang, et al. ∙

research

∙ 06/29/2020

Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction

Click-Through Rate (CTR) prediction is one of the most important machine...

0 Qingquan Song, et al. ∙

research

∙ 06/11/2020

Few-shot Neural Architecture Search

To improve the search efficiency for Neural Architecture Search (NAS), O...

0 Yiyang Zhao, et al. ∙

research

∙ 06/03/2020

FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition Function

Neural Architecture Search (NAS) yields state-of-the-art neural networks...

11 Xiaoliang Dai, et al. ∙

research

∙ 04/12/2020

FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions

Differentiable Neural Architecture Search (DNAS) has demonstrated great ...

17 Alvin Wan, et al. ∙

research

∙ 09/30/2019

Over-parameterization as a Catalyst for Better Generalization of Deep ReLU network

To analyze deep ReLU network, we adopt a student-teacher setting in whic...

37 Yuandong Tian, et al. ∙

research

∙ 09/10/2019

Bayesian Relational Memory for Semantic Visual Navigation

We introduce a new memory architecture, Bayesian Relational Memory (BRM)...

26 Yi Wu, et al. ∙

research

∙ 06/28/2019

A Neural-based Program Decompiler

Reverse engineering of binary executables is a critical problem in the c...

0 Cheng Fu, et al. ∙

research

∙ 06/17/2019

Sample-Efficient Neural Architecture Search by Learning Action Space

Neural Architecture Search (NAS) has emerged as a promising technique fo...

6 Linnan Wang, et al. ∙

research

∙ 06/06/2019

One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers

The success of lottery ticket initializations (Frankle and Carbin, 2019)...

5 Ari S. Morcos, et al. ∙

research

∙ 06/06/2019

Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP

The lottery ticket hypothesis proposes that over-parameterization of dee...

0 Haonan Yu, et al. ∙

research

∙ 06/03/2019

Hierarchical Decision Making by Generating and Following Natural Language Instructions

We explore using latent natural language instructions as an expressive a...

4 Hengyuan Hu, et al. ∙

research

∙ 05/31/2019

Luck Matters: Understanding Training Dynamics of Deep ReLU Networks

We analyze the dynamics of training deep ReLU networks and their implica...

5 Yuandong Tian, et al. ∙

Yuandong Tian

Featured Co-authors