
Numerical influence of ReLU'(0) on backpropagation
In theory, the choice of ReLU'(0) in [0, 1] for a neural network has a n...
read it

Nonsmooth Implicit Differentiation for Machine Learning and Optimization
In view of training increasingly complex learning architectures, we esta...
read it

Secondorder stepsize tuning of SGD for nonconvex optimization
In view of a direct and simple improvement of vanilla SGD, this paper pr...
read it

A Hölderian backtracking method for minmax and minmin problems
We present a new algorithm to solve minmax or minmin problems out of t...
read it

A mathematical model for automatic differentiation in machine learning
Automatic differentiation, as implemented today, does not have a simple ...
read it

Long term dynamics of the subgradient method for Lipschitz path differentiable functions
We consider the longterm dynamics of the vanishing stepsize subgradient...
read it

Optimal Complexity and Certification of Bregman FirstOrder Methods
We provide a lower bound showing that the O(1/k) convergence rate of the...
read it

Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learning
The Clarke subdifferential is not suited to tackle nonsmooth deep learni...
read it

An Inertial Newton Algorithm for Deep Learning
We devise a learning algorithm for possibly nonsmooth deep neural networ...
read it

First Order Methods beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems
We focus on nonconvex and nonsmooth minimization problems with a composi...
read it
Jérôme Bolte
is this you? claim profile