
-
Particle Dual Averaging: Optimization of Mean Field Neural Networks with Global Convergence Rate Analysis
We propose the particle dual averaging (PDA) method, which generalizes t...
read it
-
A Novel Global Spatial Attention Mechanism in Convolutional Neural Network for Medical Image Classification
Spatial attention has been introduced to convolutional neural networks (...
read it
-
Online Robust and Adaptive Learning from Data Streams
In online learning from non-stationary data streams, it is both necessar...
read it
-
Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regime
We analyze the convergence of the averaged stochastic gradient descent f...
read it
-
When Does Preconditioning Help or Hurt Generalization?
While second order optimizers such as natural gradient descent (NGD) oft...
read it
-
Exponential Convergence Rates of Classification Errors on Learning with SGD and Random Features
Although kernel methods are widely used in many learning problems, they ...
read it
-
Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space
Deep learning has exhibited superior performance for various tasks, espe...
read it
-
Data Cleansing for Models Trained with SGD
Data cleansing is a typical approach used to improve the accuracy of mac...
read it
-
Refined Generalization Analysis of Gradient Descent for Over-parameterized Two-layer Neural Networks with Smooth Activations on Classification Problems
Recently, several studies have proven the global convergence and general...
read it
-
Stochastic Gradient Descent with Exponential Convergence Rates of Expected Classification Errors
We consider stochastic gradient descent for binary classification proble...
read it
-
Functional Gradient Boosting based on Residual Network Perception
Residual Networks (ResNets) have become state-of-the-art models in deep ...
read it
-
Gradient Layer: Enhancing the Convergence of Adversarial Training for Generative Models
We propose a new technique that boosts the convergence of training gener...
read it
-
Stochastic Particle Gradient Descent for Infinite Ensembles
The superior performance of ensemble methods with infinite models are we...
read it
-
Accelerated Stochastic Gradient Descent for Minimizing Finite Sums
We propose an optimization method for minimizing the finite sums of smoo...
read it