Approximating the diagonal of a Hessian: which sample set of points should be used

04/23/2021
by   Gabriel Jarry--Bolduc, et al.
0

An explicit formula to approximate the diagonal entries of the Hessian is introduced. When the derivative-free technique called generalized centered simplex gradient is used to approximate the gradient, then the formula can be computed for only one additional function evaluation. An error bound is introduced and provides information on the form of the sample set of points that should be used to approximate the diagonal of a Hessian. If the sample set of points is built in a specific manner, it is shown that the technique is 𝒪(Δ_S^2) accurate approximation of the diagonal entries of the Hessian where Δ_S is the radius of the sample set.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/27/2012

Estimating the Hessian by Back-propagating Curvature

In this work we develop Curvature Propagation (CP), a general technique ...
research
06/01/2020

Error bounds for overdetermined and underdetermined generalized centred simplex gradients

Using the Moore–Penrose pseudoinverse, this work generalizes the gradien...
research
04/06/2023

A matrix algebra approach to approximate Hessians

This work presents a novel matrix-based method for constructing an appro...
research
06/16/2020

Practical Quasi-Newton Methods for Training Deep Neural Networks

We consider the development of practical stochastic quasi-Newton, and in...
research
04/29/2014

Fast Approximation of Rotations and Hessians matrices

A new method to represent and approximate rotation matrices is introduce...
research
08/04/2023

Exact upper bound for copulas with a given diagonal section

We answer a 15-year-old open question about the exact upper bound for bi...
research
10/15/2019

Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize

Variable metric proximal gradient (VM-PG) is a widely used class of conv...

Please sign up or login with your details

Forgot password? Click here to reset