research
∙
06/16/2023
Power-law Dynamic arising from machine learning
We study a kind of new SDE that was arisen from the research on optimiza...
share
research
∙
02/20/2023
NeuralStagger: accelerating physics-constrained neural PDE solver with spatial-temporal decomposition
Neural networks have shown great potential in accelerating the solution ...
research
∙
10/31/2022
Multilingual Speech Emotion Recognition With Multi-Gating Mechanism and Neural Architecture Search
Speech emotion recognition (SER) classifies audio into emotion categorie...
research
∙
04/13/2022
Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs
Stochastic partial differential equations (SPDEs) are significant tools ...
research
∙
10/26/2021
Equivariant vector field network for many-body system modeling
Modeling many-body systems has been a long-standing challenge in science...
research
∙
10/08/2021
Momentum Doesn't Change the Implicit Bias
The momentum acceleration technique is widely adopted in many optimizati...
research
∙
06/28/2021
R-Drop: Regularized Dropout for Neural Networks
Dropout is a powerful and widely used technique to regularize the traini...
research
∙
06/11/2021
PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Driven Adaptive Prior
Denoising diffusion probabilistic models have been recently proposed to ...
research
∙
06/08/2021
Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics
Learning dynamics governed by differential equations is crucial for pred...
research
∙
05/31/2021
Machine-Learning Non-Conservative Dynamics for New-Physics Detection
Energy conservation is a basic physics principle, the breakdown of which...
research
∙
04/11/2021
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
Transformer architecture achieves great success in abundant natural lang...
research
∙
01/08/2021
Towards Accelerating Training of Batch Normalization: A Manifold Perspective
Batch normalization (BN) has become a crucial component across diverse d...
research
∙
12/11/2020
The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks
Despite their overwhelming capacity to overfit, deep neural networks tra...
research
∙
07/01/2020
Constructing Basis Path Set by Eliminating Path Dependency
The way the basis path set works in neural network remains mysterious, a...
research
∙
06/24/2020
Dynamic of Stochastic Gradient Descent with State-Dependent Noise
Stochastic gradient descent (SGD) and its variants are mainstream method...
research
∙
10/18/2019
Interpreting Basis Path Set in Neural Networks
Based on basis path set, G-SGD algorithm significantly outperforms conve...
research
∙
03/14/2019
Reinforcement Learning with Dynamic Boltzmann Softmax Updates
Value function estimation is an important task in reinforcement learning...
research
∙
03/06/2019
Positively Scale-Invariant Flatness of ReLU Neural Networks
It was empirically confirmed by Keskar et al.SharpMinima that flatter mi...
research
∙
09/21/2018
Target Transfer Q-Learning and Its Convergence Analysis
Q-learning is one of the most popular methods in Reinforcement Learning ...
research
∙
09/19/2018
Capacity Control of ReLU Neural Networks by Basis-path Norm
Recently, path norm was proposed as a new capacity measure for neural ne...
research
∙
05/08/2018
Differential Equations for Modeling Asynchronous Algorithms
Asynchronous stochastic gradient descent (ASGD) is a popular parallel op...
research
∙
02/11/2018
Optimizing Neural Networks in the Equivalent Class Space
It has been widely observed that many activation functions and pooling m...
research
∙
09/29/2017
Convergence Analysis of Distributed Stochastic Gradient Descent with Shuffling
When using stochastic gradient descent to solve large-scale machine lear...
research
∙
09/27/2016