b'Yuanzhi Li'

research

∙ 08/04/2022

Towards Understanding Mixture of Experts in Deep Learning

The Mixture-of-Experts (MoE) layer, a sparsely-activated model controlle...

Zixiang Chen, et al. ∙

research

∙ 07/13/2022

Towards understanding how momentum improves generalization in deep learning

Stochastic gradient descent (SGD) with momentum is widely used for train...

Samy Jelassi, et al. ∙

research

∙ 05/31/2022

Learning (Very) Simple Generative Models Is Hard

Motivated by the recent empirical successes of deep generative models, w...

Sitan Chen, et al. ∙

research

∙ 05/12/2022

The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning

Recently the surprising discovery of the Bootstrap Your Own Latent (BYOL...

Zixin Wen, et al. ∙

research

∙ 04/24/2022

Complete Policy Regret Bounds for Tallying Bandits

Policy regret is a well established notion of measuring the performance ...

Dhruv Malik, et al. ∙

research

∙ 04/08/2022

Learning Polynomial Transformations

We consider the problem of learning high dimensional polynomial transfor...

Sitan Chen, et al. ∙

research

∙ 01/18/2022

Minimax Optimality (Probably) Doesn't Imply Distribution Learning for GANs

Arguably the most fundamental question in the theory of generative adver...

Sitan Chen, et al. ∙

research

∙ 11/01/2021

Posterior Inference for Quantile Regression: Adaptation to Sparsity

Quantile regression is a powerful data analysis tool that accommodates h...

Yuanzhi Li, et al. ∙

research

∙ 11/01/2021

Settling the Horizon-Dependence of Sample Complexity in Reinforcement Learning

Recently there is a surge of interest in understanding the horizon-depen...

Yuanzhi Li, et al. ∙

research

∙ 09/29/2021

On the One-sided Convergence of Adam-type Algorithms in Non-convex Non-concave Min-max Optimization

Adam-type methods, the extension of adaptive gradient methods, have show...

Zehao Dou, et al. ∙

research

∙ 08/25/2021

Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization

Adaptive gradient methods such as Adam have gained increasing popularity...

Difan Zou, et al. ∙

research

∙ 06/17/2021

LoRA: Low-Rank Adaptation of Large Language Models

The dominant paradigm of natural language processing consists of large-s...

Edward J. Hu, et al. ∙

research

∙ 06/15/2021

Sample Efficient Reinforcement Learning In Continuous State Spaces: A Perspective Beyond Linearity

Reinforcement learning (RL) is empirically successful in complex nonline...

Dhruv Malik, et al. ∙

research

∙ 06/04/2021

Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions

Generative adversarial networks (GANs) are among the most successful mod...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 05/31/2021

Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning

How can neural networks trained by contrastive learning extract features...

Zixin Wen, et al. ∙

research

∙ 02/26/2021

Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability

We empirically demonstrate that full-batch gradient descent on neural ne...

Jeremy M Cohen, et al. ∙

research

∙ 01/01/2021

When Is Generalizable Reinforcement Learning Tractable?

Agents trained by reinforcement learning (RL) often fail to generalize b...

Dhruv Malik, et al. ∙

research

∙ 12/17/2020

Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning

We formally study how Ensemble of deep learning models can improve test ...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 09/30/2020

A law of robustness for two-layers neural networks

We initiate the study of the inherent tradeoffs between the size of a ne...

Sébastien Bubeck, et al. ∙

research

∙ 07/09/2020

Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK

We consider the dynamic of gradient descent for learning a two-layer neu...

Yuanzhi Li, et al. ∙

research

∙ 05/20/2020

Feature Purification: How Adversarial Training Performs Robust Deep Learning

Despite the great empirical success of adversarial training to defend de...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 03/09/2020

When can Wasserstein GANs minimize Wasserstein Distance?

Generative Adversarial Networks (GANs) are widely used models to learn c...

Yuanzhi Li, et al. ∙

research

∙ 01/13/2020

Backward Feature Correction: How Deep Learning Performs Deep Learning

How does a 110-layer ResNet learn a high-complexity classifier using rel...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 07/10/2019

Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks

Stochastic gradient descent with a large initial learning rate is a wide...

Yuanzhi Li, et al. ∙

research

∙ 06/25/2019

Complexity of Highly Parallel Non-Smooth Convex Optimization

A landmark result of non-smooth convex optimization is that gradient des...

Sébastien Bubeck, et al. ∙

research

∙ 05/24/2019

What Can ResNet Learn Efficiently, Going Beyond Kernels?

How can neural networks such as ResNet efficiently learn CIFAR-10 with t...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 04/28/2019

Non-Stochastic Multi-Player Multi-Armed Bandits: Optimal Rate With Collision Information, Sublinear Without

We consider the non-stochastic version of the (cooperative) multi-player...

Sébastien Bubeck, et al. ∙

research

∙ 02/04/2019

Can SGD Learn Recurrent Neural Networks with Provable Generalization?

Recurrent Neural Networks (RNNs) are among the most popular models in se...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 01/29/2019

Improved Path-length Regret Bounds for Bandits

We study adaptive regret bounds in terms of the variation of the losses ...

Sébastien Bubeck, et al. ∙

research

∙ 11/12/2018

Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers

Neural networks have great success in many machine learning applications...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 11/09/2018

A Convergence Theory for Deep Learning via Over-Parameterization

Deep neural networks (DNNs) have demonstrated dominating performance in ...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 11/02/2018

Chasing Nested Convex Bodies Nearly Optimally

The convex body chasing problem, introduced by Friedman and Linial, is a...

Sébastien Bubeck, et al. ∙

research

∙ 11/02/2018

Competitively Chasing Convex Bodies

Let F be a family of sets in some metric space. In the F-chasing problem...

Sébastien Bubeck, et al. ∙

research

∙ 10/29/2018

On the Convergence Rate of Training Recurrent Neural Networks

Despite the huge success of deep learning, our understanding to how the ...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 10/09/2018

Statistical Convergence of the EM Algorithm on Gaussian Mixture Models

We study the convergence behavior of the Expectation Maximization (EM) a...

Ruofei Zhao, et al. ∙

research

∙ 08/03/2018

Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data

Neural networks have many successful applications, while much less theor...

Yuanzhi Li, et al. ∙

research

∙ 07/10/2018

Algorithmic Framework for Model-based Reinforcement Learning with Theoretical Guarantees

While model-based reinforcement learning has empirically been shown to s...

Huazhe Xu, et al. ∙

research

∙ 06/08/2018

The Well Tempered Lasso

We study the complexity of the entire regularization path for least squa...

Yuanzhi Li, et al. ∙

research

∙ 04/20/2018

Online Improper Learning with an Approximation Oracle

We revisit the question of reducing online learning to approximate optim...

Elad Hazan, et al. ∙

research

∙ 04/03/2018

Operator Scaling via Geodesically Convex Optimization, Invariant Theory and Polynomial Identity Testing

We propose a new second-order method for geodesically convex optimizatio...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 02/22/2018

Learning Mixtures of Linear Regressions with Nearly Optimal Complexity

Mixtures of Linear Regressions (MLR) is an important mixture model with ...

Yuanzhi Li, et al. ∙

research

∙ 02/17/2018

An Alternative View: When Does SGD Escape Local Minima?

Stochastic gradient descent (SGD) is widely used in machine learning. Al...

Robert Kleinberg, et al. ∙

research

∙ 02/09/2018

Make the Minority Great Again: First-Order Regret Bound for Contextual Bandits

Regret bounds in online learning compare the player's performance to L^*...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 12/26/2017

Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations

We show that the (stochastic) gradient descent algorithm provides an imp...

Yuanzhi Li, et al. ∙

research

∙ 12/26/2017

Algorithmic Regularization in Over-parameterized Matrix Recovery

We study the problem of recovering a low-rank matrix X^ from linear meas...

Yuanzhi Li, et al. ∙

research

∙ 11/17/2017

Neon2: Finding Local Minima via First-Order Oracles

We propose a reduction for non-convex optimization that can (1) turn a s...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 11/14/2017

Near-Optimal Discrete Optimization for Experimental Design: A Regret Minimization Approach

The experimental design problem concerns the selection of k points from ...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 11/03/2017

An homotopy method for ℓ_p regression provably beyond self-concordance and in input-sparsity time

We consider the problem of linear regression where the ℓ_2^n norm loss (...

Sébastien Bubeck, et al. ∙

research

∙ 08/07/2017

Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls

We propose a rank-k variant of the classical Frank-Wolfe algorithm to so...

Zeyuan Allen-Zhu, et al. ∙

research

∙ 07/25/2017

A Nearly Instance Optimal Algorithm for Top-k Ranking under the Multinomial Logit Model

We study the active learning problem of top-k ranking from multi-wise co...

Xi Chen, et al. ∙

Yuanzhi Li

Featured Co-authors