Frederik Kunstner | DeepAI

AI Chat

Featured Co-authors

Philipp Hennig
82 publications
Simon Lacoste-Julien
79 publications
Mark Schmidt
54 publications
Mohammad Emtiyaz Khan
37 publications
Issam Laradji
27 publications
Sharan Vaswani
19 publications
Damien Scieur
14 publications
Didrik Nielsen
10 publications
Lukas Balles
10 publications
Jonathan Wilder Lavington
9 publications
Felix Dangel
6 publications

∙ 06/05/2023

Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking

The backtracking line-search is an effective technique to automatically ...

0 Frederik Kunstner, et al. ∙

∙ 04/27/2023

Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be

The success of the Adam optimizer on a wide array of architectures has m...

0 Frederik Kunstner, et al. ∙

∙ 11/12/2021

Convergence Rates for the MAP of an Exponential Family and Stochastic Mirror Descent – an Open Problem

We consider the problem of upper bounding the expected log-likelihood su...

0 Rémi Le Priol, et al. ∙

∙ 11/02/2020

Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent

Expectation maximization (EM) is the default algorithm for fitting proba...

0 Frederik Kunstner, et al. ∙

∙ 06/11/2020

Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search)

As adaptive gradient methods are typically used for training over-parame...

12 Sharan Vaswani, et al. ∙

∙ 12/23/2019

BackPACK: Packing more into backprop

Automatic differentiation frameworks are optimized for exactly one thing...

10 Felix Dangel, et al. ∙

∙ 05/29/2019

Limitations of the Empirical Fisher Approximation

Natural gradient descent, which preconditions a gradient descent update ...

0 Frederik Kunstner, et al. ∙

∙ 11/11/2018

SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient

Uncertainty estimation in large deep-learning models is a computationall...

0 Aaron Mishkin, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world