Simon S. Du

is this you? claim profile

∙ 0 followers
Institute for Advanced Study ∙ Carnegie Mellon University ∙ University of Washington

Research Intern at facebook 2017, Research Intern at Microsoft 2016, Consulting Intern at Accenture 2015, Research Assistant at UC Berkeley from 2013-2014, Software Engineering Intern at Google 2014, Research Assistant at Bay Area Intellectual Property Group 2013, Consulting Intern at CCID Consulting Co. 2012, PhD in Machine Learning Department at Carnegie Mellon University 2015-2020

Featured Co-authors

Michael I. Jordan
168 publications
Xi Chen
128 publications
Ruslan Salakhutdinov
123 publications
Li Fei-Fei
99 publications
Yi Zhang
97 publications
Barnabas Poczos
92 publications
Joshua B. Tenenbaum
79 publications
Liwei Wang
69 publications
Quanquan Gu
59 publications
Aarti Singh
56 publications
Bo Liu
56 publications

Few-Shot Learning via Learning the Representation, Provably

This paper studies few-shot learning via representation learning, where ...
02/21/2020 ∙ by Simon S. Du, et al. ∙ 46 ∙ share

read it
Gradient Descent Provably Optimizes Over-parameterized Neural Networks

One of the mystery in the success of neural networks is randomly initial...
10/04/2018 ∙ by Simon S. Du, et al. ∙ 30 ∙ share

read it
Gradient Descent Finds Global Minima of Deep Neural Networks

Gradient descent finds a global minimum in training deep neural networks...
11/09/2018 ∙ by Simon S. Du, et al. ∙ 20 ∙ share

read it
Enhanced Convolutional Neural Tangent Kernels

Recent research shows that for training with ℓ_2 loss, convolutional neu...
11/03/2019 ∙ by Zhiyuan Li, et al. ∙ 17 ∙ share

read it
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks

Recent works have cast some light on the mystery of why deep nets fit an...
01/24/2019 ∙ by Sanjeev Arora, et al. ∙ 16 ∙ share

read it
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks

Recent research shows that the following two models are equivalent: (a) ...
10/03/2019 ∙ by Sanjeev Arora, et al. ∙ 16 ∙ share

read it
Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?

Modern deep learning methods provide an effective means to learn good re...
10/07/2019 ∙ by Simon S. Du, et al. ∙ 15 ∙ share

read it
Dual Sequential Monte Carlo: Tunneling Filtering and Planning in Continuous POMDPs

We present the DualSMC network that solves continuous POMDPs by learning...
09/28/2019 ∙ by Yunbo Wang, et al. ∙ 15 ∙ share

read it
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality

Adversarial training is a popular method to give neural nets robustness ...
02/16/2020 ∙ by Yi Zhang, et al. ∙ 11 ∙ share

read it
Is Long Horizon Reinforcement Learning More Difficult Than Short Horizon Reinforcement Learning?

Learning to plan for long horizons is a central challenge in episodic re...
05/01/2020 ∙ by Ruosong Wang, et al. ∙ 7 ∙ share

read it
Global Convergence of Adaptive Gradient Methods for An Over-parameterized Neural Network

Adaptive gradient methods like AdaGrad are widely used in optimizing neu...
02/19/2019 ∙ by Xiaoxia Wu, et al. ∙ 6 ∙ share

read it
Provable Representation Learning for Imitation Learning via Bi-level Optimization

A common strategy in modern learning systems is to learn a representatio...
02/24/2020 ∙ by Sanjeev Arora, et al. ∙ 6 ∙ share

read it
Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels

While graph kernels (GKs) are easy to train and enjoy provable theoretic...
05/30/2019 ∙ by Simon S. Du, et al. ∙ 4 ∙ share

read it
Towards Understanding the Importance of Shortcut Connections in Residual Networks

Residual Network (ResNet) is undoubtedly a milestone in deep learning. R...
09/10/2019 ∙ by Tianyi Liu, et al. ∙ 2 ∙ share

read it
Provably Efficient Exploration for RL with Unsupervised Learning

We study how to use unsupervised learning for efficient exploration in r...
03/15/2020 ∙ by Fei Feng, et al. ∙ 2 ∙ share

read it
When is Particle Filtering Efficient for POMDP Sequential Planning?

Particle filtering is a popular method for inferring latent states in st...
06/10/2020 ∙ by Simon S. Du, et al. ∙ 1 ∙ share

read it
When is a Convolutional Filter Easy To Learn?

We analyze the convergence of (stochastic) gradient descent algorithm fo...
09/18/2017 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Gradient Descent Can Take Exponential Time to Escape Saddle Points

Although gradient descent (GD) almost always escapes saddle points asymp...
05/29/2017 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Stochastic Variance Reduction Methods for Policy Evaluation

Policy evaluation is a crucial step in many reinforcement-learning proce...
02/25/2017 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Computationally Efficient Robust Estimation of Sparse Functionals

Many conventional statistical procedures are extremely sensitive to seem...
02/24/2017 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
On the Power of Truncated SVD for General High-rank Matrix Estimation Problems

We show that given an estimate A that is close to a general high-rank po...
02/22/2017 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Efficient Nonparametric Smoothness Estimation

Sobolev quantities (norms, inner products, and distances) of probability...
05/19/2016 ∙ by Shashank Singh, et al. ∙ 0 ∙ share

read it
An Improved Gap-Dependency Analysis of the Noisy Power Method

We consider the noisy power method algorithm, which has wide application...
02/23/2016 ∙ by Maria-Florina Balcan, et al. ∙ 0 ∙ share

read it
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima

We consider the problem of learning a one-hidden-layer neural network wi...
12/03/2017 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Linear Convergence of the Primal-Dual Gradient Method for Convex-Concave Saddle Point Problems without Strong Convexity

We consider the convex-concave saddle point problem _x_y f(x)+y^ A x-g(y...
02/05/2018 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Fast and Sample Efficient Inductive Matrix Completion via Multi-Phase Procrustes Flow

We revisit the inductive matrix completion problem that aims to recover ...
03/03/2018 ∙ by Xiao Zhang, et al. ∙ 0 ∙ share

read it
On the Power of Over-parametrization in Neural Networks with Quadratic Activation

We provide new theoretical insights on why over-parametrization is effec...
03/03/2018 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Improved Learning of One-hidden-layer Convolutional Neural Networks with Overlaps

We propose a new algorithm to learn a one-hidden-layer convolutional neu...
05/20/2018 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
How Many Samples are Needed to Learn a Convolutional Neural Network?

A widespread folklore for explaining the success of convolutional neural...
05/21/2018 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced

We study the implicit regularization imposed by gradient descent for lea...
06/04/2018 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Robust Nonparametric Regression under Huber's ε-contamination Model

We consider the non-parametric regression problem under Huber's ϵ-contam...
05/26/2018 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Near-Linear Time Local Polynomial Nonparametric Estimation

Local polynomial regression (Fan & Gijbels, 1996) is an important class ...
02/26/2018 ∙ by Yining Wang, et al. ∙ 0 ∙ share

read it
Understanding the Acceleration Phenomenon via High-Resolution Differential Equations

Gradient-based optimization algorithms can be studied from the perspecti...
10/21/2018 ∙ by Bin Shi, et al. ∙ 0 ∙ share

read it
Width Provably Matters in Optimization for Deep Linear Neural Networks

We prove that for an L-layer fully-connected linear neural network, if t...
01/24/2019 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Provably efficient RL with Rich Observations via Latent State Decoding

We study the exploration problem in episodic MDPs with rich observations...
01/25/2019 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Acceleration via Symplectic Discretization of High-Resolution Differential Equations

We study first-order optimization methods obtained by discretizing ordin...
02/11/2019 ∙ by Bin Shi, et al. ∙ 0 ∙ share

read it
Hitting Time of Stochastic Gradient Langevin Dynamics to Stationary Points: A Direct Analysis

Stochastic gradient Langevin dynamics (SGLD) is a fundamental algorithm ...
04/30/2019 ∙ by Xi Chen, et al. ∙ 0 ∙ share

read it
What Can Neural Networks Reason About?

Neural networks have successfully been applied to solving reasoning task...
05/30/2019 ∙ by Keyulu Xu, et al. ∙ 0 ∙ share

read it
On Exact Computation with an Infinitely Wide Neural Net

How well does a classic deep net architecture like AlexNet or VGG19 clas...
04/26/2019 ∙ by Sanjeev Arora, et al. ∙ 0 ∙ share

read it
Provably Efficient Q-learning with Function Approximation via Distribution Shift Error Checking Oracle

Q-learning with function approximation is one of the most popular method...
06/14/2019 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Continuous Control with Contexts, Provably

A fundamental challenge in artificial intelligence is to build an agent ...
10/30/2019 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Optimism in Reinforcement Learning with Generalized Linear Function Approximation

We design a new provably efficient algorithm for episodic reinforcement ...
12/09/2019 ∙ by Yining Wang, et al. ∙ 0 ∙ share

read it
Agnostic Q-learning with Function Approximation in Deterministic Systems: Tight Bounds on Approximation Error and Sample Complexity

The current paper studies the problem of agnostic Q-learning with functi...
02/17/2020 ∙ by Simon S. Du, et al. ∙ 0 ∙ share

read it
Q-learning with Logarithmic Regret

This paper presents the first non-asymptotic result showing that a model...
06/16/2020 ∙ by Kunhe Yang, et al. ∙ 0 ∙ share

read it
On Reward-Free Reinforcement Learning with Linear Function Approximation

Reward-free reinforcement learning (RL) is a framework which is suitable...
06/19/2020 ∙ by Ruosong Wang, et al. ∙ 0 ∙ share

read it