research
∙
05/23/2023
One-step differentiation of iterative algorithms
In appropriate frameworks, automatic differentiation is transparent to t...
share
research
∙
12/15/2022
Differentiating Nonsmooth Solutions to Parametric Monotone Inclusion Problems
We leverage path differentiability and a recent result on nonsmooth impl...
research
∙
07/26/2022
The derivatives of Sinkhorn-Knopp converge
We show that the derivatives of the Sinkhorn-Knopp algorithm, or iterati...
research
∙
06/01/2022
Nonsmooth automatic differentiation: a cheap gradient principle and other complexity results
We provide a simple model to estimate the computational costs of the bac...
research
∙
05/31/2022
Automatic differentiation of nonsmooth iterative algorithms
Differentiation along algorithms, i.e., piggyback propagation of derivat...
research
∙
01/11/2022
Path differentiability of ODE flows
We consider flows of ordinary differential equations (ODEs) driven by pa...
research
∙
06/23/2021
Numerical influence of ReLU'(0) on backpropagation
In theory, the choice of ReLU'(0) in [0, 1] for a neural network has a n...
research
∙
06/08/2021
Nonsmooth Implicit Differentiation for Machine Learning and Optimization
In view of training increasingly complex learning architectures, we esta...
research
∙
03/05/2021
Second-order step-size tuning of SGD for non-convex optimization
In view of a direct and simple improvement of vanilla SGD, this paper pr...
research
∙
11/24/2020
Sequential convergence of AdaGrad algorithm for smooth convex optimization
We prove that the iterates produced by, either the scalar step size vari...
research
∙
07/17/2020
A Hölderian backtracking method for min-max and min-min problems
We present a new algorithm to solve min-max or min-min problems out of t...
research
∙
07/15/2020
Incremental Without Replacement Sampling in Nonconvex Optimization
Minibatch decomposition methods for empirical risk minimization are comm...
research
∙
06/03/2020
A mathematical model for automatic differentiation in machine learning
Automatic differentiation, as implemented today, does not have a simple ...
research
∙
05/29/2020
Long term dynamics of the subgradient method for Lipschitz path differentiable functions
We consider the long-term dynamics of the vanishing stepsize subgradient...
research
∙
02/10/2020
Polynomial Optimization for Bounding Lipschitz Constants of Deep Networks
The Lipschitz constant of a network plays an important role in many appl...
research
∙
10/31/2019
Rate of convergence for geometric inference based on the empirical Christoffel function
We consider the problem of estimating the support of a measure from a fi...
research
∙
09/23/2019
Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learning
The Clarke subdifferential is not suited to tackle nonsmooth deep learni...
research
∙
05/29/2019
An Inertial Newton Algorithm for Deep Learning
We devise a learning algorithm for possibly nonsmooth deep neural networ...
research
∙
10/19/2018
Data analysis from empirical moments and the Christoffel function
Spectral features of the empirical moment matrix constitute a resourcefu...
research
∙
05/21/2018