DeepAI AI Chat
Log In Sign Up

Reverse-Mode Automatic Differentiation of Compiled Programs

by   Max Aehle, et al.

Tools for algorithmic differentiation (AD) provide accurate derivatives of computer-implemented functions for use in, e. g., optimization and machine learning (ML). However, they often require the source code of the function to be available in a restricted set of programming languages. As a step towards making AD accessible for code bases with cross-language or closed-source components, we recently presented the forward-mode AD tool Derivgrind. It inserts forward-mode AD logic into the machine code of a compiled program using the Valgrind dynamic binary instrumentation framework. This work extends Derivgrind, adding the capability to record the real-arithmetic evaluation tree, and thus enabling operator overloading style reverse-mode AD for compiled programs. We maintain the high level of correctness reported for Derivgrind's forward mode, failing the same few testcases in an extensive test suite for the same well-understood reasons. Runtime-wise, the recording slows down the execution of a compiled 64-bit benchmark program by a factor of about 180.


page 1

page 2

page 3

page 4


Decomposing reverse-mode automatic differentiation

We decompose reverse-mode automatic differentiation into (forward-mode) ...

Forward-Mode Automatic Differentiation of Compiled Programs

Algorithmic differentiation (AD) is a set of techniques to obtain accura...

Automatic Differentiation in ROOT

In mathematics and computer algebra, automatic differentiation (AD) is a...

Binomial Checkpointing for Arbitrary Programs with No User Annotation

Heretofore, automatic checkpointing at procedure-call boundaries, to red...

Divide-and-Conquer Checkpointing for Arbitrary Programs with No User Annotation

Classical reverse-mode automatic differentiation (AD) imposes only a sma...

Automatic Differentiation of Programs with Discrete Randomness

Automatic differentiation (AD), a technique for constructing new program...

High-Performance Derivative Computations using CoDiPack

There are several AD tools available, which all implement different stra...