research
∙
07/18/2023
Optimistic Estimate Uncovers the Potential of Nonlinear Models
We propose an optimistic estimate to evaluate the best possible fitting ...
share
research
∙
05/25/2023
Stochastic Modified Equations and Dynamics of Dropout Algorithm
Dropout is a widely utilized regularization technique in the training of...
research
∙
05/20/2023
Loss Spike in Training Neural Networks
In this work, we study the mechanism underlying loss spikes observed dur...
research
∙
05/17/2023
Understanding the Initial Condensation of Convolutional Neural Networks
Previous research has shown that fully-connected networks with small ini...
research
∙
04/03/2023
Laplace-fPINNs: Laplace-based fractional physics-informed neural networks for solving forward and inverse problems of subdiffusion
The use of Physics-informed neural networks (PINNs) has shown promise in...
research
∙
03/12/2023
Phase Diagram of Initial Condensation for Two-layer Neural Networks
The phenomenon of distinct behaviors exhibited by neural networks under ...
research
∙
11/22/2022
Bayesian Inversion with Neural Operator (BINO) for Modeling Subdiffusion: Forward and Inverse Problems
Fractional diffusion equations have been an effective tool for modeling ...
research
∙
11/21/2022
Linear Stability Hypothesis and Rank Stratification for Nonlinear Models
Models with nonlinear architectures/parameterizations such as deep neura...
research
∙
07/13/2022
Implicit regularization of dropout
It is important to understand how the popular regularization method drop...
research
∙
05/26/2022
Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks
Unraveling the general structure underlying the loss landscapes of deep ...
research
∙
05/25/2022
An Experimental Comparison Between Temporal Difference and Residual Gradient with Neural Network Approximation
Gradient descent or its variants are popular in training neural networks...
research
∙
05/24/2022
Empirical Phase Diagram for Three-layer Neural Networks with Infinite Width
Substantial work indicates that the dynamics of neural networks (NNs) is...
research
∙
01/28/2022
Limitation of characterizing implicit regularization by data-independent functions
In recent years, understanding the implicit regularization of neural net...
research
∙
01/19/2022
Overview frequency principle/spectral bias in deep learning
Understanding deep learning is increasingly emergent as it penetrates mo...
research
∙
01/09/2022
A multi-scale sampling method for accurate and robust deep neural network to predict combustion chemical kinetics
Machine learning has long been considered as a black box for predicting ...
research
∙
01/06/2022
A deep learning-based model reduction (DeePMR) method for simplifying chemical kinetics
A deep learning-based model reduction (DeePMR) method for simplifying ch...
research
∙
12/10/2021
Subspace Decomposition based DNN algorithm for elliptic-type multi-scale PDEs
While deep learning algorithms demonstrate a great potential in scientif...
research
∙
11/30/2021
Embedding Principle: a hierarchical structure of loss landscape of deep neural networks
We prove a general Embedding Principle of loss landscape of deep neural ...
research
∙
11/01/2021
A variance principle explains why dropout finds flatter minima
Although dropout has achieved great success in deep learning, little is ...
research
∙
07/17/2021
Data-informed Deep Optimization
Complex design problems are common in the scientific and industrial fiel...
research
∙
07/13/2021
Force-in-domain GAN inversion
Empirical works suggest that various semantics emerge in the latent spac...
research
∙
07/08/2021
MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs
In this paper, we propose a model-operator-data network (MOD-Net) for so...
research
∙
05/30/2021
Embedding Principle of Loss Landscape of Deep Neural Networks
Understanding the structure of loss landscape of deep neural networks (D...
research
∙
05/25/2021
Towards Understanding the Condensation of Two-layer Neural Networks at Initial Training
It is important to study what implicit regularization is imposed on the ...
research
∙
05/25/2021
An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Network
Deep neural network (DNN) usually learns the target function from low to...
research
∙
01/30/2021
Linear Frequency Principle Model to Understand the Absence of Overfitting in Neural Networks
Why heavily parameterized neural networks (NNs) do not overfit the data ...
research
∙
01/04/2021
Frequency Principle in Deep Learning Beyond Gradient-descent-based Training
Frequency perspective recently makes progress in understanding deep lear...
research
∙
12/06/2020
Fourier-domain Variational Formulation and Its Well-posedness for Supervised Learning
A supervised learning problem is to find a function in a hypothesis func...
research
∙
10/15/2020
On the exact computation of linear frequency principle dynamics and its generalization
Recent works show an intriguing phenomenon of Frequency Principle (F-Pri...
research
∙
07/29/2020
A regularized deep matrix factorized model of matrix completion for image restoration
It has been an important approach of using matrix completion to perform ...
research
∙
07/28/2020
Deep frequency principle towards understanding why deeper learning is faster
Understanding the effect of depth in deep learning is a critical problem...
research
∙
07/22/2020
Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains
In this paper, we propose novel multi-scale DNNs (MscaleDNN) using the i...
research
∙
07/15/2020
Phase diagram for two-layer ReLU neural networks at infinite-width limit
How neural network behaves during the training over different choices of...
research
∙
02/19/2020
Implicit bias with Ritz-Galerkin method in understanding deep learning for solving PDEs
This paper aims at studying the difference between Ritz-Galerkin (R-G) m...
research
∙
12/06/2019
A priori generalization error for two-layer ReLU neural network through minimum norm solution
We focus on estimating a priori generalization error of two-layer ReLU n...
research
∙
10/25/2019
Multi-scale Deep Neural Networks for Solving High Dimensional PDEs
In this paper, we propose the idea of radial scaling in frequency domain...
research
∙
06/21/2019
Theory of the Frequency Principle for General Deep Neural Networks
Along with fruitful applications of Deep Neural Networks (DNNs) to reali...
research
∙
05/24/2019
Explicitizing an Implicit Bias of the Frequency Principle in Two-layer Neural Networks
It remains a puzzle that why deep neural networks (DNNs), with more para...
research
∙
05/19/2019
A type of generalization error induced by initialization in deep neural networks
How different initializations and loss functions affect the learning of ...
research
∙
01/19/2019
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
We study the training process of Deep Neural Networks (DNNs) from the Fo...
research
∙
11/26/2018
Frequency Principle in Deep Learning with General Loss Functions and Its Potential Application
Previous studies have shown that deep neural networks (DNNs) with common...
research
∙
11/11/2018
Swift Two-sample Test on High-dimensional Neural Spiking Data
To understand how neural networks process information, it is important t...
research
∙
08/30/2018