Global linear convergence of Newton's method without strong-convexity or Lipschitz gradients

06/01/2018
by   Sai Praneeth Karimireddy, et al.
0

We show that Newton's method converges globally at a linear rate for objective functions whose Hessians are stable. This class of problems includes many functions which are not strongly convex, such as logistic regression. Our linear convergence result is (i) affine-invariant, and holds even if an (ii) approximate Hessian is used, and if the subproblems are (iii) only solved approximately. Thus we theoretically demonstrate the superiority of Newton's method over first-order methods, which would only achieve a sublinear O(1/t^2) rate under similar conditions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/03/2021

Regularized Newton Method with Global O(1/k^2) Convergence

We present a Newton-type method that converges fast from any initializat...
research
09/30/2018

Newton-MR: Newton's Method Without Smoothness or Convexity

Establishing global convergence of the classical Newton's method has lon...
research
02/18/2020

Distributed Adaptive Newton Methods with Globally Superlinear Convergence

This paper considers the distributed optimization problem over a network...
research
11/10/2022

A Randomised Subspace Gauss-Newton Method for Nonlinear Least-Squares

We propose a Randomised Subspace Gauss-Newton (R-SGN) algorithm for solv...
research
02/17/2020

A Newton Frank-Wolfe Method for Constrained Self-Concordant Minimization

We demonstrate how to scalably solve a class of constrained self-concord...
research
11/19/2020

On the asymptotic rate of convergence of Stochastic Newton algorithms and their Weighted Averaged versions

The majority of machine learning methods can be regarded as the minimiza...
research
07/05/2019

On stable invertibility and global Newton convergence for convex monotonic functions

We derive a simple criterion that ensures uniqueness, Lipschitz stabilit...

Please sign up or login with your details

Forgot password? Click here to reset