On the Computational Complexity of the Chain Rule of Differential Calculus

07/12/2021
by   Uwe Naumann, et al.
0

Many modern numerical methods in computational science and engineering rely on derivatives of mathematical models for the phenomena under investigation. The computation of these derivatives often represents the bottleneck in terms of overall runtime performance. First and higher derivative tensors need to be evaluated efficiently. The chain rule of differentiation is the fundamental prerequisite for computing accurate derivatives of composite functions which perform a potentially very large number of elemental function evaluations. Data flow dependences amongst the elemental functions give rise to a combinatorial optimization problem. We formulate Chain Rule Differentiation and we prove it to be NP-complete. Pointers to research on its approximate solution are given.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/17/2021

Hessian Chain Bracketing

Second derivatives of mathematical models for real-world phenomena are f...
research
07/02/2022

Combinatory Adjoints and Differentiation

We develop a compositional approach for automatic and symbolic different...
research
11/29/2019

Chain Rules for Hessian and Higher Derivatives Made Easy by Tensor Calculus

Computing multivariate derivatives of matrix-like expressions in the com...
research
05/11/2023

Differentiable Programming: Efficient Smoothing of Control-Flow-Induced Discontinuities

We want to obtain derivatives in discontinuous program code, where defau...
research
03/12/2020

Optimization of Generalized Jacobian Chain Products without Memory Constraints

The efficient computation of Jacobians represents a fundamental challeng...
research
03/08/2020

Keeping it simple: Implementation and performance of the proto-principle of adaptation and learning in the language sciences

In this paper we present the Widrow-Hoff rule and its applications to la...
research
03/07/2017

Mini-symposium on automatic differentiation and its applications in the financial industry

Automatic differentiation is involved for long in applied mathematics as...

Please sign up or login with your details

Forgot password? Click here to reset