DeepAI AI Chat
Log In Sign Up

Adjoint-based exact Hessian-vector multiplication using symplectic Runge–Kutta methods

by   Shin-ichi Ito, et al.

We consider a function of the numerical solution of an initial value problem, its Hessian matrix with respect to the initial data, and the computation of a Hessian-vector multiplication. A simple way of approximating the Hessian-vector multiplication is to integrate the so-called second-order adjoint system numerically. However, the error in the approximation could be significant unless the numerical integration is sufficiently accurate. This paper presents a novel algorithm that computes the intended Hessian-vector multiplication exactly. For this aim, we give a new concise derivation of the second-order adjoint system and show that the intended multiplication can be computed exactly by applying a particular numerical method to the second-order adjoint system. In the discussion, symplectic partitioned Runge–Kutta methods play an important role.


page 1

page 2

page 3

page 4


HesScale: Scalable Computation of Hessian Diagonals

Second-order optimization uses curvature information about the objective...

Generalization of partitioned Runge–Kutta methods for adjoint systems

This study computes the gradient of a function of numerical solutions of...

On backpropagating Hessians through ODEs

We discuss the problem of numerically backpropagating Hessians through o...

The Hypervolume Indicator Hessian Matrix: Analytical Expression, Computational Time Complexity, and Sparsity

The problem of approximating the Pareto front of a multiobjective optimi...

A spectral characterization and an approximation scheme for the Hessian eigenvalue

We revisit the k-Hessian eigenvalue problem on a smooth, bounded, (k-1)-...

Mathematical derivation for Vora-Value based filter design method: Gradient and Hessian

In this paper, we present the detailed mathematical derivation of the gr...

Generalizing and Improving Jacobian and Hessian Regularization

Jacobian and Hessian regularization aim to reduce the magnitude of the f...