A frequency-domain analysis of inexact gradient descent

12/31/2019
by   Oran Gannot, et al.
0

We study robustness properties of inexact gradient descent for strongly convex functions, as well as for the larger class of functions with sector-bounded gradients, under a relative error model. Proofs of the corresponding convergence rates are based on frequency-domain criteria for the stability of nonlinear systems perturbed by additive noise.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/09/2020

Perturbed gradient descent with occupation time

This paper develops further the idea of perturbed gradient descent, by a...
research
06/26/2018

A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updates

We provide tight finite-time convergence bounds for gradient descent and...
research
11/03/2020

The Complexity of Gradient Descent: CLS = PPAD ∩ PLS

We study search problems that can be solved by performing Gradient Desce...
research
08/10/2022

Frequency propagation: Multi-mechanism learning in nonlinear physical networks

We introduce frequency propagation, a learning algorithm for nonlinear p...
research
09/11/2020

Stability of Decentralized Gradient Descent in Open Multi-Agent Systems

The aim of decentralized gradient descent (DGD) is to minimize a sum of ...
research
10/26/2022

Sinusoidal Frequency Estimation by Gradient Descent

Sinusoidal parameter estimation is a fundamental task in applications fr...
research
11/25/2021

A Letter on Convergence of In-Parameter-Linear Nonlinear Neural Architectures with Gradient Learnings

This letter summarizes and proves the concept of bounded-input bounded-s...

Please sign up or login with your details

Forgot password? Click here to reset