
Particle Dual Averaging: Optimization of Mean Field Neural Networks with Global Convergence Rate Analysis
We propose the particle dual averaging (PDA) method, which generalizes t...
read it

A Novel Global Spatial Attention Mechanism in Convolutional Neural Network for Medical Image Classification
Spatial attention has been introduced to convolutional neural networks (...
read it

Online Robust and Adaptive Learning from Data Streams
In online learning from nonstationary data streams, it is both necessar...
read it

Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regime
We analyze the convergence of the averaged stochastic gradient descent f...
read it

When Does Preconditioning Help or Hurt Generalization?
While second order optimizers such as natural gradient descent (NGD) oft...
read it

Exponential Convergence Rates of Classification Errors on Learning with SGD and Random Features
Although kernel methods are widely used in many learning problems, they ...
read it

Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space
Deep learning has exhibited superior performance for various tasks, espe...
read it

Data Cleansing for Models Trained with SGD
Data cleansing is a typical approach used to improve the accuracy of mac...
read it

Refined Generalization Analysis of Gradient Descent for Overparameterized Twolayer Neural Networks with Smooth Activations on Classification Problems
Recently, several studies have proven the global convergence and general...
read it

Stochastic Gradient Descent with Exponential Convergence Rates of Expected Classification Errors
We consider stochastic gradient descent for binary classification proble...
read it

Functional Gradient Boosting based on Residual Network Perception
Residual Networks (ResNets) have become stateoftheart models in deep ...
read it

Gradient Layer: Enhancing the Convergence of Adversarial Training for Generative Models
We propose a new technique that boosts the convergence of training gener...
read it

Stochastic Particle Gradient Descent for Infinite Ensembles
The superior performance of ensemble methods with infinite models are we...
read it

Accelerated Stochastic Gradient Descent for Minimizing Finite Sums
We propose an optimization method for minimizing the finite sums of smoo...
read it
Atsushi Nitanda
is this you? claim profile