
Stochastic Gradient Descent Escapes Saddle Points Efficiently
This paper considers the perturbed stochastic gradient descent algorithm...
02/13/2019 ∙ by Chi Jin, et al. ∙ 20 ∙ shareread it

A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm
In this note, we derive concentration inequalities for random vectors wi...
02/11/2019 ∙ by Chi Jin, et al. ∙ 16 ∙ shareread it

The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure
There is a stark disparity between the step size schedules used in pract...
04/29/2019 ∙ by Rong Ge, et al. ∙ 12 ∙ shareread it

Explaining Landscape Connectivity of Lowcost Solutions for Multilayer Nets
Mode connectivity is a surprising phenomenon in the loss landscape of de...
06/14/2019 ∙ by Rohith Kuditipudi, et al. ∙ 3 ∙ shareread it

Beyond Logconcavity: Provable Guarantees for Sampling Multimodal Distributions using Simulated Tempering Langevin Monte Carlo
A key task in Bayesian statistics is sampling from distributions that ar...
10/07/2017 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

On the Optimization Landscape of Tensor Decompositions
Nonconvex optimization with local search heuristics has been widely use...
06/18/2017 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis
In this paper we develop a new framework that captures the common landsc...
04/03/2017 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

How to Escape Saddle Points Efficiently
This paper shows that a perturbed form of gradient descent converges to ...
03/02/2017 ∙ by Chi Jin, et al. ∙ 0 ∙ shareread it

Generalization and Equilibrium in Generative Adversarial Nets (GANs)
We show that training of generative adversarial network (GAN) may not ha...
03/02/2017 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ shareread it

Provable learning of Noisyor Networks
Many machine learning applications use latent variable models to explain...
12/28/2016 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ shareread it

Homotopy Analysis for Tensor PCA
Developing efficient and guaranteed nonconvex algorithms has been an imp...
10/28/2016 ∙ by Anima Anandkumar, et al. ∙ 0 ∙ shareread it

Provable Algorithms for Inference in Topic Models
Recently, there has been considerable progress on designing algorithms w...
05/27/2016 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ shareread it

Matrix Completion has No Spurious Local Minimum
Matrix completion is a basic machine learning problem that has wide appl...
05/24/2016 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

Efficient Algorithms for Largescale Generalized Eigenvector Computation and Canonical Correlation Analysis
This paper considers the problem of canonicalcorrelation analysis (CCA)...
04/13/2016 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

Efficient approaches for escaping higher order saddle points in nonconvex optimization
Local search heuristics for nonconvex optimizations are popular in appl...
02/18/2016 ∙ by Anima Anandkumar, et al. ∙ 0 ∙ shareread it

Intersecting Faces: Nonnegative Matrix Factorization With New Guarantees
Nonnegative matrix factorization (NMF) is a natural model of admixture ...
07/08/2015 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

Unregularizing: approximate proximal point and faster stochastic algorithms for empirical risk minimization
We develop a family of accelerated stochastic algorithms that minimize s...
06/24/2015 ∙ by Roy Frostig, et al. ∙ 0 ∙ shareread it

Decomposing Overcomplete 3rd Order Tensors using SumofSquares Algorithms
Tensor rank and lowrank tensor decompositions have many applications in...
04/21/2015 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

Escaping From Saddle Points  Online Stochastic Gradient for Tensor Decomposition
We analyze stochastic gradient descent for optimizing nonconvex functio...
03/06/2015 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

Simple, Efficient, and Neural Algorithms for Sparse Coding
Sparse coding is a basic task in many fields including signal processing...
03/02/2015 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ shareread it

Competing with the Empirical Risk Minimizer in a Single Pass
In many estimation problems, e.g. linear and logistic regression, we wis...
12/20/2014 ∙ by Roy Frostig, et al. ∙ 0 ∙ shareread it

Analyzing Tensor Power Method Dynamics in Overcomplete Regime
We present a novel analysis of the dynamics of tensor power iterations i...
11/06/2014 ∙ by Anima Anandkumar, et al. ∙ 0 ∙ shareread it

Sample Complexity Analysis for Learning Overcomplete Latent Variable Models through Tensor Methods
We provide guarantees for learning latent variable models emphasizing on...
08/03/2014 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

Guaranteed NonOrthogonal Tensor Decomposition via Alternating Rank1 Updates
In this paper, we provide local and global convergence guarantees for re...
02/21/2014 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

More Algorithms for Provable Dictionary Learning
In dictionary learning, also known as sparse coding, the algorithm is gi...
01/03/2014 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ shareread it

New Algorithms for Learning Incoherent and Overcomplete Dictionaries
In sparse recovery we are given a matrix A (the dictionary) and a vector...
08/28/2013 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ shareread it

A Tensor Approach to Learning Mixed Membership Community Models
Community detection is the task of detecting hidden communities from obs...
02/12/2013 ∙ by Anima Anandkumar, et al. ∙ 0 ∙ shareread it

A Practical Algorithm for Topic Modeling with Provable Guarantees
Topic models provide a useful method for dimensionality reduction and ex...
12/19/2012 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ shareread it

Tensor decompositions for learning latent variable models
This work considers a computationally and statistically efficient parame...
10/29/2012 ∙ by Anima Anandkumar, et al. ∙ 0 ∙ shareread it

Learning Onehiddenlayer Neural Networks with Landscape Design
We consider the problem of learning a onehiddenlayer neural network: w...
11/01/2017 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

Global Convergence of Policy Gradient Methods for Linearized Control Problems
Direct policy gradient methods for reinforcement learning and continuous...
01/15/2018 ∙ by Maryam Fazel, et al. ∙ 0 ∙ shareread it

Stronger generalization bounds for deep nets via a compression approach
Deep nets generalize well despite having more parameters than the number...
02/14/2018 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ shareread it

DynIMS: A Dynamic Memory Controller for Inmemory Storage on HPC Systems
In order to boost the performance of dataintensive computing on HPC sys...
09/29/2016 ∙ by Pengfei Xuan, et al. ∙ 0 ∙ shareread it

Minimizing Nonconvex Population Risk from Rough Empirical Risk
Population riskthe expectation of the loss over the sampling mechanis...
03/25/2018 ∙ by Chi Jin, et al. ∙ 0 ∙ shareread it

NonConvex Matrix Completion Against a SemiRandom Adversary
Matrix completion is a wellstudied problem with many machine learning a...
03/28/2018 ∙ by Yu Cheng, et al. ∙ 0 ∙ shareread it

Learning Twolayer Neural Networks with Symmetric Inputs
We give a new algorithm for learning a twolayer neural network under a ...
10/16/2018 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

HighDimensional Robust Mean Estimation in NearlyLinear Time
We study the fundamental problem of highdimensional mean estimation in ...
11/23/2018 ∙ by Yu Cheng, et al. ∙ 0 ∙ shareread it

Simulated Tempering Langevin Monte Carlo II: An Improved Proof using Soft Markov Chain Decomposition
A key task in Bayesian machine learning is sampling from distributions t...
11/29/2018 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

Understanding Composition of Word Embeddings via Tensor Decomposition
Word embedding is a powerful tool in natural language processing. In thi...
02/02/2019 ∙ by Abraham Frandsen, et al. ∙ 0 ∙ shareread it

Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization
Variance reduction techniques like SVRG provide simple and fast algorith...
05/01/2019 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

Faster Algorithms for HighDimensional Robust Covariance Estimation
We study the problem of estimating the covariance matrix of a highdimen...
06/11/2019 ∙ by Yu Cheng, et al. ∙ 0 ∙ shareread it

Mildly Overparametrized Neural Nets can Memorize Training Data Efficiently
It has been observed zhang2016understanding that deep neural networks ca...
09/26/2019 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it

Estimating Normalizing Constants for LogConcave Distributions: Algorithms and Lower Bounds
Estimating the normalizing constant of an unnormalized probability distr...
11/08/2019 ∙ by Rong Ge, et al. ∙ 0 ∙ shareread it
Rong Ge
is this you? claim profile
Assistant Professor at the Computer Science Department of Duke University.