
Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning
It is not clear yet why ADAMalike adaptive gradient algorithms suffer f...
read it

Interpretable Neural Networks for Panel Data Analysis in Economics
The lack of interpretability and transparency are preventing economists ...
read it

The Knowledge Graph for Macroeconomic Analysis with Alternative Big Data
The current knowledge system of macroeconomics is built on interactions ...
read it

A priori estimates for classification problems using neural networks
We consider binary and multiclass classification problems using hypothe...
read it

Machine Learning and Computational Mathematics
Neural networkbased machine learning is capable of approximating functi...
read it

Towards a Mathematical Understanding of Neural NetworkBased Machine Learning: what we know and what we don't
The purpose of this article is to review the achievements made in the la...
read it

On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysis
We study the approximation properties and optimization dynamics of recur...
read it

A Qualitative Study of the Dynamic Behavior of Adaptive Gradient Algorithms
The dynamic behavior of RMSprop and Adam algorithms is studied through a...
read it

OnsagerNet: Learning Stable and Interpretable Dynamics using a Generalized Onsager Principle
We propose a systematic method for learning stable and interpretable dyn...
read it

Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
In recent years, tremendous progress has been made on numerical algorith...
read it

The Slow Deterioration of the Generalization Error of the Random Feature Model
The random feature model exhibits a kind of resonance behavior when the ...
read it

DeePKS: a comprehensive datadriven approach towards chemically accurate density functional theory
We propose a general machine learningbased framework for building an ac...
read it

On the Banach spaces associated with multilayer ReLU networks: Function representation, approximation theory and gradient descent dynamics
We develop Banach spaces for ReLU neural networks of finite depth L and ...
read it

Coarsegrained spectral projection (CGSP): A scalable and parallelizable deep learningbased approach to quantum unitary dynamics
We propose the coarsegrained spectral projection method (CGSP), a deep ...
read it

The QuenchingActivation Behavior of the Gradient Descent Dynamics for Twolayer Neural Network Models
A numerical and phenomenological study of the gradient descent (GD) algo...
read it

Representation formulas and pointwise properties for Barron functions
We study the natural function space for infinitely wide twolayer neural...
read it

Integrating Machine Learning with PhysicsBased Modeling
Machine learning is poised as a very powerful tool that can drastically ...
read it

Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective
We prove that the gradient descent training of a twolayer neural networ...
read it

Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels
We establish a scale separation of Kolmogorov width type between subspac...
read it

86 PFLOPS Deep Potential Molecular Dynamics simulation of 100 million atoms with ab initio accuracy
We present the GPU version of DeePMDkit, which, upon training a deep ne...
read it

Machine learning based nonNewtonian fluid model with molecular fidelity
We introduce a machinelearningbased framework for constructing continu...
read it

Machine Learning from a Continuous Viewpoint
We present a continuous formulation of machine learning, as a problem in...
read it

On the Generalization Properties of Minimumnorm Solutions for Overparameterized Neural Network Models
We study the generalization properties of minimumnorm solutions for thr...
read it

A Mathematical Model for Linguistic Universals
Inspired by chemical kinetics and neurobiology, we propose a mathematica...
read it

Deep neural network for Wannier function centers
We introduce a deep neural network (DNN) model that assigns the position...
read it

Barron Spaces and the Compositional Function Spaces for Neural Network Models
One of the key issues in the analysis of machine learning models is to i...
read it

Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skipconnections
The behavior of the gradient descent (GD) algorithm is analyzed for a de...
read it

A Comparative Analysis of the Optimization and Generalization Property of Twolayer Neural Network and Random Feature Models Under Gradient Descent Dynamics
A fairly comprehensive analysis is presented for the gradient descent dy...
read it

A Priori Estimates of the Population Risk for Residual Networks
Optimal a priori estimates are derived for the population risk of a regu...
read it

Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations
We develop the mathematical foundations of the stochastic modified equat...
read it

Active Learning of Uniformly Accurate Interatomic Potentials for Materials Simulation
An active learning procedure called Deep Potential Generator (DPGEN) is...
read it

A Priori Estimates of the Generalization Error for Twolayer Neural Networks
New estimates for the generalization error are established for the twol...
read it

MongeAmpère Flow for Generative Modeling
We present a deep generative model, named MongeAmpère flow, which build...
read it

Model Reduction with Memory and the Machine Learning of Dynamical Systems
The wellknown MoriZwanzig theory tells us that model reduction leads t...
read it

A MeanField Optimal Control Formulation of Deep Learning
Recent work linking deep neural networks and dynamical systems opened up...
read it

Exponential Convergence of the Deep Neural Network Approximation for Analytic Functions
We prove that for analytic functions in low dimension, the convergence r...
read it

Understanding and Enhancing the Transferability of Adversarial Examples
Stateoftheart deep neural networks are known to be vulnerable to adve...
read it

DeePMDkit: A deep learning package for manybody potential energy representation and molecular dynamics
Recent developments in manybody potential energy representation via dee...
read it

Reinforced dynamics for enhanced sampling in large atomic and molecular systems. I. Basic Methodology
A new approach for efficiently exploring the configuration space and com...
read it

Maximum Principle Based Algorithms for Deep Learning
The continuous dynamical system approach to deep learning is explored in...
read it

The Deep Ritz method: A deep learningbased numerical algorithm for solving variational problems
We propose a deep learning based method, the Deep Ritz Method, for numer...
read it

Machine learning approximation algorithms for highdimensional fully nonlinear partial differential equations and secondorder backward stochastic differential equations
Highdimensional partial differential equations (PDE) appear in a number...
read it

Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes
It is widely observed that deep learning models with learned parameters ...
read it

Deep learningbased numerical methods for highdimensional parabolic partial differential equations and backward stochastic differential equations
We propose a new algorithm for solving parabolic partial differential eq...
read it

Deep Learning Approximation for Stochastic Control Problems
Many real world stochastic control problems suffer from the "curse of di...
read it

Stochastic modified equations and adaptive stochastic gradient algorithms
We develop the method of stochastic modified equations (SME), in which s...
read it

Convolutional neural networks with lowrank regularization
Large CNNs have delivered impressive performance in various computer vis...
read it

Functional FrankWolfe Boosting for General Loss Functions
Boosting is a generic learning method for classification and regression....
read it

Multiscale Adaptive Representation of Signals: I. The Basic Framework
We introduce a framework for designing multiscale, adaptive, shiftinva...
read it
Weinan E
is this you? claim profile
Professor, Department of Mathematics and Program in Applied and Computational Mathematics at Princeton University