Yuanzhi Li

is this you? claim profile

∙ 0 followers
MIT ∙ Princeton University ∙ Facebook ∙ Microsoft ∙ Stanford University

PhD Student at Princeton University

LoRA: Low-Rank Adaptation of Large Language Models

The dominant paradigm of natural language processing consists of large-s...
06/17/2021 ∙ by Edward J. Hu, et al. ∙ 13 ∙ share

read it
Sample Efficient Reinforcement Learning In Continuous State Spaces: A Perspective Beyond Linearity

Reinforcement learning (RL) is empirically successful in complex nonline...
06/15/2021 ∙ by Dhruv Malik, et al. ∙ 2 ∙ share

read it
Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions

Generative adversarial networks (GANs) are among the most successful mod...
06/04/2021 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning

How can neural networks trained by contrastive learning extract features...
05/31/2021 ∙ by Zixin Wen, et al. ∙ 0 ∙ share

read it
Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability

We empirically demonstrate that full-batch gradient descent on neural ne...
02/26/2021 ∙ by Jeremy M Cohen, et al. ∙ 0 ∙ share

read it
When Is Generalizable Reinforcement Learning Tractable?

Agents trained by reinforcement learning (RL) often fail to generalize b...
01/01/2021 ∙ by Dhruv Malik, et al. ∙ 0 ∙ share

read it
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning

We formally study how Ensemble of deep learning models can improve test ...
12/17/2020 ∙ by Zeyuan Allen-Zhu, et al. ∙ 78 ∙ share

read it
A law of robustness for two-layers neural networks

We initiate the study of the inherent tradeoffs between the size of a ne...
09/30/2020 ∙ by Sébastien Bubeck, et al. ∙ 0 ∙ share

read it
Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK

We consider the dynamic of gradient descent for learning a two-layer neu...
07/09/2020 ∙ by Yuanzhi Li, et al. ∙ 8 ∙ share

read it
Feature Purification: How Adversarial Training Performs Robust Deep Learning

Despite the great empirical success of adversarial training to defend de...
05/20/2020 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
When can Wasserstein GANs minimize Wasserstein Distance?

Generative Adversarial Networks (GANs) are widely used models to learn c...
03/09/2020 ∙ by Yuanzhi Li, et al. ∙ 0 ∙ share

read it
Backward Feature Correction: How Deep Learning Performs Deep Learning

How does a 110-layer ResNet learn a high-complexity classifier using rel...
01/13/2020 ∙ by Zeyuan Allen-Zhu, et al. ∙ 5 ∙ share

read it
Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks

Stochastic gradient descent with a large initial learning rate is a wide...
07/10/2019 ∙ by Yuanzhi Li, et al. ∙ 6 ∙ share

read it
Complexity of Highly Parallel Non-Smooth Convex Optimization

A landmark result of non-smooth convex optimization is that gradient des...
06/25/2019 ∙ by Sébastien Bubeck, et al. ∙ 0 ∙ share

read it
What Can ResNet Learn Efficiently, Going Beyond Kernels?

How can neural networks such as ResNet efficiently learn CIFAR-10 with t...
05/24/2019 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Non-Stochastic Multi-Player Multi-Armed Bandits: Optimal Rate With Collision Information, Sublinear Without

We consider the non-stochastic version of the (cooperative) multi-player...
04/28/2019 ∙ by Sébastien Bubeck, et al. ∙ 4 ∙ share

read it
Can SGD Learn Recurrent Neural Networks with Provable Generalization?

Recurrent Neural Networks (RNNs) are among the most popular models in se...
02/04/2019 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Improved Path-length Regret Bounds for Bandits

We study adaptive regret bounds in terms of the variation of the losses ...
01/29/2019 ∙ by Sébastien Bubeck, et al. ∙ 8 ∙ share

read it
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers

Neural networks have great success in many machine learning applications...
11/12/2018 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
A Convergence Theory for Deep Learning via Over-Parameterization

Deep neural networks (DNNs) have demonstrated dominating performance in ...
11/09/2018 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Chasing Nested Convex Bodies Nearly Optimally

The convex body chasing problem, introduced by Friedman and Linial, is a...
11/02/2018 ∙ by Sébastien Bubeck, et al. ∙ 0 ∙ share

read it
Competitively Chasing Convex Bodies

Let F be a family of sets in some metric space. In the F-chasing problem...
11/02/2018 ∙ by Sébastien Bubeck, et al. ∙ 0 ∙ share

read it
On the Convergence Rate of Training Recurrent Neural Networks

Despite the huge success of deep learning, our understanding to how the ...
10/29/2018 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Statistical Convergence of the EM Algorithm on Gaussian Mixture Models

We study the convergence behavior of the Expectation Maximization (EM) a...
10/09/2018 ∙ by Ruofei Zhao, et al. ∙ 0 ∙ share

read it
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data

Neural networks have many successful applications, while much less theor...
08/03/2018 ∙ by Yuanzhi Li, et al. ∙ 0 ∙ share

read it
Algorithmic Framework for Model-based Reinforcement Learning with Theoretical Guarantees

While model-based reinforcement learning has empirically been shown to s...
07/10/2018 ∙ by Huazhe Xu, et al. ∙ 0 ∙ share

read it
The Well Tempered Lasso

We study the complexity of the entire regularization path for least squa...
06/08/2018 ∙ by Yuanzhi Li, et al. ∙ 0 ∙ share

read it
Online Improper Learning with an Approximation Oracle

We revisit the question of reducing online learning to approximate optim...
04/20/2018 ∙ by Elad Hazan, et al. ∙ 0 ∙ share

read it
Operator Scaling via Geodesically Convex Optimization, Invariant Theory and Polynomial Identity Testing

We propose a new second-order method for geodesically convex optimizatio...
04/03/2018 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Learning Mixtures of Linear Regressions with Nearly Optimal Complexity

Mixtures of Linear Regressions (MLR) is an important mixture model with ...
02/22/2018 ∙ by Yuanzhi Li, et al. ∙ 0 ∙ share

read it
An Alternative View: When Does SGD Escape Local Minima?

Stochastic gradient descent (SGD) is widely used in machine learning. Al...
02/17/2018 ∙ by Robert Kleinberg, et al. ∙ 0 ∙ share

read it
Make the Minority Great Again: First-Order Regret Bound for Contextual Bandits

Regret bounds in online learning compare the player's performance to L^*...
02/09/2018 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations

We show that the (stochastic) gradient descent algorithm provides an imp...
12/26/2017 ∙ by Yuanzhi Li, et al. ∙ 0 ∙ share

read it
Algorithmic Regularization in Over-parameterized Matrix Recovery

We study the problem of recovering a low-rank matrix X^ from linear meas...
12/26/2017 ∙ by Yuanzhi Li, et al. ∙ 0 ∙ share

read it
Neon2: Finding Local Minima via First-Order Oracles

We propose a reduction for non-convex optimization that can (1) turn a s...
11/17/2017 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Near-Optimal Discrete Optimization for Experimental Design: A Regret Minimization Approach

The experimental design problem concerns the selection of k points from ...
11/14/2017 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
An homotopy method for ℓ_p regression provably beyond self-concordance and in input-sparsity time

We consider the problem of linear regression where the ℓ_2^n norm loss (...
11/03/2017 ∙ by Sébastien Bubeck, et al. ∙ 0 ∙ share

read it
Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls

We propose a rank-k variant of the classical Frank-Wolfe algorithm to so...
08/07/2017 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
A Nearly Instance Optimal Algorithm for Top-k Ranking under the Multinomial Logit Model

We study the active learning problem of top-k ranking from multi-wise co...
07/25/2017 ∙ by Xi Chen, et al. ∙ 0 ∙ share

read it
Provable Alternating Gradient Descent for Non-negative Matrix Factorization with Strong Correlations

Non-negative matrix factorization is a basic tool for decomposing data i...
06/13/2017 ∙ by Yuanzhi Li, et al. ∙ 0 ∙ share

read it
Follow the Compressed Leader: Faster Online Learning of Eigenvectors and Faster MMWU

The online problem of computing the top eigenvector is fundamental to ma...
01/06/2017 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Recovery Guarantee of Non-negative Matrix Factorization via Alternating Updates

Non-negative matrix factorization is a popular tool for decomposing data...
11/11/2016 ∙ by Yuanzhi Li, et al. ∙ 0 ∙ share

read it
Faster Principal Component Regression and Stable Matrix Chebyshev Approximation

We solve principal component regression (PCR), up to a multiplicative ac...
08/16/2016 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
First Efficient Convergence for Streaming k-PCA: a Global, Gap-Free, and Near-Optimal Rate

We study streaming principal component analysis (PCA), that is to find, ...
07/26/2016 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Doubly Accelerated Methods for Faster CCA and Generalized Eigendecomposition

We study k-GenEV, the problem of finding the top k generalized eigenvect...
07/20/2016 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
LazySVD: Even Faster SVD Decomposition Yet Without Agonizing Pain

We study k-SVD that is to obtain the first k singular vectors of a matri...
07/12/2016 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Approximate maximum entropy principles via Goemans-Williamson with applications to provable variational methods

The well known maximum-entropy principle due to Jaynes, which states tha...
07/12/2016 ∙ by Yuanzhi Li, et al. ∙ 0 ∙ share

read it
Recovery guarantee of weighted low-rank approximation via alternating minimization

Many applications require recovering a ground truth low-rank matrix from...
02/06/2016 ∙ by Yuanzhi Li, et al. ∙ 0 ∙ share

read it
Linear Algebraic Structure of Word Senses, with Applications to Polysemy

Word embeddings are ubiquitous in NLP and information retrieval, but it'...
01/14/2016 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ share

read it
RAND-WALK: A Latent Variable Model Approach to Word Embeddings

Semantic word embeddings represent the meaning of a word via a vector, a...
02/12/2015 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ share

read it

Yuanzhi Li

Featured Co-authors