research
∙
07/24/2023
Nonparametric Linear Feature Learning in Regression Through Regularisation
Representation learning plays a crucial role in automated feature select...
share
research
∙
07/04/2023
Generalization Guarantees via Algorithm-dependent Rademacher Complexity
Algorithm- and data-dependent generalization bounds are required to expl...
research
∙
06/13/2023
Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGD
Neural network compression has been an increasingly important subject, d...
research
∙
06/07/2023
Learning via Wasserstein-Based High Probability Generalisation Bounds
Minimising upper bounds on the population risk or the generalisation gap...
research
∙
05/20/2023
Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent
Algorithmic stability is an important notion that has proven powerful fo...
research
∙
03/30/2023
Efficient Sampling of Stochastic Differential Equations with Positive Semi-Definite Models
This paper deals with the problem of efficient sampling from a stochasti...
research
∙
02/10/2023
Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD
Cyclic and randomized stepsizes are widely used in the deep learning pra...
research
∙
02/06/2023
Generalization Bounds with Data-dependent Fractal Dimensions
Providing generalization guarantees for modern neural networks has been ...
research
∙
01/27/2023
Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions
Heavy-tail phenomena in stochastic gradient descent (SGD) have been repo...
research
∙
09/19/2022
Generalization Bounds for Stochastic Gradient Descent via Localized ε-Covers
In this paper, we propose a new covering technique localized for the tra...
research
∙
06/02/2022
Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares
Recent studies have shown that heavy tails can emerge in stochastic opti...
research
∙
05/23/2022
Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descent
Recent studies have shown that gradient descent (GD) can achieve improve...
research
∙
05/13/2022
Heavy-Tail Phenomenon in Decentralized SGD
Recent theoretical studies have shown that heavy-tails can emerge in sto...
research
∙
03/04/2022
Rate-Distortion Theoretic Generalization Bounds for Stochastic Learning Algorithms
Understanding generalization in modern machine learning settings has bee...
research
∙
11/25/2021
Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks
Disobeying the classical wisdom of statistical learning theory, modern d...
research
∙
08/02/2021
Generalization Properties of Stochastic Optimizers via Trajectory Analysis
Despite the ubiquitous use of stochastic optimization algorithms in mach...
research
∙
06/29/2021
Fast Approximation of the Sliced-Wasserstein Distance Using Concentration of Random Projections
The Sliced-Wasserstein distance (SW) is being increasingly used in machi...
research
∙
06/09/2021
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms
Understanding generalization in deep learning has been one of the major ...
research
∙
06/07/2021
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks
Neural network compression techniques have become increasingly popular a...
research
∙
05/18/2021
Relative Positional Encoding for Transformers with Linear Complexity
Recent advances in Transformer models allow for unprecedented sequence l...
research
∙
02/20/2021
Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance
Recent studies have provided both empirical and theoretical evidence ill...
research
∙
02/13/2021
Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections
Gaussian noise injections (GNIs) are a family of simple and widely-used ...
research
∙
02/10/2021
Self-Supervised VQ-VAE For One-Shot Music Style Transfer
Neural style transfer, allowing to apply the artistic style of one image...
research
∙
07/14/2020
Explicit Regularisation in Gaussian Noise Injections
We study the regularisation induced in neural networks by Gaussian noise...
research
∙
07/13/2020
Quantitative Propagation of Chaos for SGD in Wide Neural Networks
In this paper, we investigate the limiting behavior of a continuous-time...
research
∙
06/16/2020
Hausdorff Dimension, Stochastic Differential Equations, and Generalization in Neural Networks
Despite its success in a wide range of applications, characterizing the ...
research
∙
06/08/2020
The Heavy-Tail Phenomenon in SGD
In recent years, various notions of capacity and complexity have been pr...
research
∙
04/01/2020
Synchronizing Probability Measures on Rotations via Optimal Transport
We introduce a new paradigm, measure synchronization, for synchronizing ...
research
∙
03/12/2020
Statistical and Topological Properties of Sliced Probability Divergences
The idea of slicing divergences has been proven to be successful when co...
research
∙
02/28/2020
Generalized Sliced Distances for Probability Distributions
Probability metrics have become an indispensable part of modern statisti...
research
∙
02/13/2020
Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise
Stochastic gradient descent with momentum (SGDm) is one of the most popu...
research
∙
11/29/2019
On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks
The gradient noise (GN) in the stochastic gradient descent (SGD) algorit...
research
∙
10/28/2019
Approximate Bayesian Computation with the Sliced-Wasserstein Distance
Approximate Bayesian Computation (ABC) is a popular method for approxima...
research
∙
07/04/2019
Supervised Symbolic Music Style Translation Using Synthetic Data
Research on style transfer and domain translation has clearly demonstrat...
research
∙
06/21/2019
First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise
Stochastic gradient descent (SGD) has been widely used in machine learni...
research
∙
06/11/2019
Asymptotic Guarantees for Learning Generative Models with the Sliced-Wasserstein Distance
Minimum expected distance estimation (MEDE) algorithms have been widely ...
research
∙
04/11/2019
Probabilistic Permutation Synchronization using the Riemannian Structure of the Birkhoff Polytope
We present an entirely new geometric and probabilistic approach to synch...
research
∙
03/11/2019
Bayesian Allocation Model: Inference by Sequential Monte Carlo for Nonnegative Tensor Factorizations and Topic Models using Polya Urns
We introduce a dynamic generative model, Bayesian allocation model (BAM)...
research
∙
02/08/2019
Speech enhancement with variational autoencoders and alpha-stable distributions
This paper focuses on single-channel semi-supervised speech enhancement....
research
∙
02/01/2019
Generalized Sliced Wasserstein Distances
The Wasserstein distance and its variations, e.g., the sliced-Wasserstei...
research
∙
01/22/2019
Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization
Recent studies on diffusion-based sampling methods have shown that Lange...
research
∙
01/18/2019
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks
The gradient noise (GN) in the stochastic gradient descent (SGD) algorit...
research
∙
06/21/2018
Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions
By building up on the recent theory that established the connection betw...
research
∙
06/07/2018
Asynchronous Stochastic Quasi-Newton MCMC for Non-Convex Optimization
Recent studies have illustrated that stochastic gradient Markov Chain Mo...
research
∙
05/31/2018
Bayesian Pose Graph Optimization via Bingham Distributions and Tempered Geodesic MCMC
We introduce Tempered Geodesic Markov Chain Monte Carlo (TG-MCMC) algori...
research
∙
02/26/2018
A Generative Model for Non-Intrusive Load Monitoring in Commercial Buildings
In the recent years, there has been an increasing academic and industria...
research
∙
06/12/2017
Fractional Langevin Monte Carlo: Exploring Lévy Driven Stochastic Differential Equations for Markov Chain Monte Carlo
Along with the recent advances in scalable Markov Chain Monte Carlo meth...
research
∙
05/22/2017
Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding
Neural time-series data contain a wide variety of prototypical signal wa...
research
∙
02/10/2016
Stochastic Quasi-Newton Langevin Monte Carlo
Recently, Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods...
research
∙
09/05/2015