A matrix algebra approach to approximate Hessians

04/06/2023
by   W. Hare, et al.
0

This work presents a novel matrix-based method for constructing an approximation Hessian using only function evaluations. The method requires less computational power than interpolation-based methods and is easy to implement in matrix-based programming languages such as MATLAB. As only function evaluations are required, the method is suitable for use in derivative-free algorithms. For reasonably structured sample sets, the method is proven to create an order-1 accurate approximation of the full Hessian. Under more specialized structures, the method is proved to yield order-2 accuracy. The undetermined case, where the number of sample points is less than required for full interpolation, is studied and error bounds are developed for the resulting partial Hessians.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/05/2023

First and zeroth-order implementations of the regularized Newton method with lazy approximated Hessians

In this work, we develop first-order (Hessian-free) and zero-order (deri...
research
04/23/2021

Approximating the diagonal of a Hessian: which sample set of points should be used

An explicit formula to approximate the diagonal entries of the Hessian i...
research
12/22/2019

Modeling Hessian-vector products in nonlinear optimization: New Hessian-free methods

In this paper, we suggest two ways of calculating interpolation models f...
research
07/17/2022

SP2: A Second Order Stochastic Polyak Method

Recently the "SP" (Stochastic Polyak step size) method has emerged as a ...
research
07/07/2023

Point spread function approximation of high rank Hessians with locally supported non-negative integral kernels

We present an efficient matrix-free point spread function (PSF) method f...
research
08/04/2021

An estimate of approximation of an analytic function of a matrix by a rational function

Let A be a square complex matrix; z_1, ..., z_N∈ℂ be arbitrary (possibly...
research
10/27/2019

Fast Evaluation and Approximation of the Gauss-Newton Hessian Matrix for the Multilayer Perceptron

We introduce a fast algorithm for entry-wise evaluation of the Gauss-New...

Please sign up or login with your details

Forgot password? Click here to reset