research
∙
07/11/2022
Wavelet Conditional Renormalization Group
We develop a multiscale approach to estimate high-dimensional probabilit...
share
research
∙
02/09/2022
Optimal learning rate schedules in high-dimensional non-convex optimization problems
Learning rate schedules are ubiquitously used to speed up and improve op...
research
∙
06/10/2021
Transformed CNNs: recasting pre-trained convolutional layers with self-attention
Vision Transformers (ViT) have recently emerged as a powerful alternativ...
research
∙
04/27/2021
Sifting out the features by pruning: Are convolutional networks the winning lottery ticket of fully connected ones?
Pruning methods can considerably reduce the size of artificial neural ne...
research
∙
03/19/2021
ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
Convolutional architectures have proven extremely successful for vision ...
research
∙
03/09/2021
More data or more parameters? Investigating the effect of data structure on generalization
One of the central features of deep learning is the generalization abili...
research
∙
06/19/2020
An analytic theory of shallow networks dynamics for hinge loss classification
Neural networks have been shown to perform incredibly well in classifica...
research
∙
06/12/2020
Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval
Despite the widespread use of gradient-based algorithms for optimizing h...
research
∙
06/05/2020
Triple descent and the two kinds of overfitting: Where why do they appear?
A recent line of research has highlighted the existence of a double desc...
research
∙
03/02/2020
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime
Deep neural networks can achieve remarkable generalization performances ...
research
∙
12/04/2019
Landscape Complexity for the Empirical Risk of Generalized Linear Models
We present a method to obtain the average and the typical value of the n...
research
∙
07/18/2019
Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Model
Gradient-based algorithms are effective for many machine learning tasks,...
research
∙
06/16/2019
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias
Despite the phenomenal success of deep neural networks in a broad range ...
research
∙
06/03/2019
Attractive vs. truncated repulsive supercooled liquids : dynamics is encoded in the pair correlation function
We compare glassy dynamics in two liquids that differ in the form of the...
research
∙
05/29/2019
How to iron out rough landscapes and get optimal performances: Replicated Gradient Descent and its application to tensor PCA
In many high-dimensional estimation problems the main task consists in m...
research
∙
04/03/2019
Large deviations for the largest eigenvalues and eigenvectors of spiked random matrices
We consider matrices formed by a random N× N matrix drawn from the Gauss...
research
∙
01/06/2019
Scaling description of generalization with number of parameters in deep learning
We provide a description for the evolution of the generalization perform...
research
∙
12/21/2018
Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference
Gradient-descent-based algorithms and their stochastic versions have wid...
research
∙
10/22/2018
A jamming transition from under- to over-parametrization affects loss landscape and generalization
We argue that in fully-connected networks a phase transition delimits th...
research
∙
09/25/2018
The jamming transition as a paradigm to understand the loss landscape of deep neural networks
Deep learning has been immensely successful at a variety of tasks, rangi...
research
∙
04/08/2018