research
∙
05/23/2023
One-step differentiation of iterative algorithms
In appropriate frameworks, automatic differentiation is transparent to t...
share
research
∙
12/15/2022
Differentiating Nonsmooth Solutions to Parametric Monotone Inclusion Problems
We leverage path differentiability and a recent result on nonsmooth impl...
research
∙
06/01/2022
Nonsmooth automatic differentiation: a cheap gradient principle and other complexity results
We provide a simple model to estimate the computational costs of the bac...
research
∙
05/31/2022
Automatic differentiation of nonsmooth iterative algorithms
Differentiation along algorithms, i.e., piggyback propagation of derivat...
research
∙
06/23/2021
Numerical influence of ReLU'(0) on backpropagation
In theory, the choice of ReLU'(0) in [0, 1] for a neural network has a n...
research
∙
06/08/2021
Nonsmooth Implicit Differentiation for Machine Learning and Optimization
In view of training increasingly complex learning architectures, we esta...
research
∙
03/05/2021
Second-order step-size tuning of SGD for non-convex optimization
In view of a direct and simple improvement of vanilla SGD, this paper pr...
research
∙
07/17/2020
A Hölderian backtracking method for min-max and min-min problems
We present a new algorithm to solve min-max or min-min problems out of t...
research
∙
06/03/2020
A mathematical model for automatic differentiation in machine learning
Automatic differentiation, as implemented today, does not have a simple ...
research
∙
05/29/2020
Long term dynamics of the subgradient method for Lipschitz path differentiable functions
We consider the long-term dynamics of the vanishing stepsize subgradient...
research
∙
11/19/2019
Optimal Complexity and Certification of Bregman First-Order Methods
We provide a lower bound showing that the O(1/k) convergence rate of the...
research
∙
09/23/2019
Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learning
The Clarke subdifferential is not suited to tackle nonsmooth deep learni...
research
∙
05/29/2019
An Inertial Newton Algorithm for Deep Learning
We devise a learning algorithm for possibly nonsmooth deep neural networ...
research
∙
06/20/2017