DeepAI AI Chat
Log In Sign Up

Binomial Checkpointing for Arbitrary Programs with No User Annotation

11/10/2016
by   Jeffrey Mark Siskind, et al.
pearlmutter.net
0

Heretofore, automatic checkpointing at procedure-call boundaries, to reduce the space complexity of reverse mode, has been provided by systems like Tapenade. However, binomial checkpointing, or treeverse, has only been provided in Automatic Differentiation (AD) systems in special cases, e.g., through user-provided pragmas on DO loops in Tapenade, or as the nested taping mechanism in adol-c for time integration processes, which requires that user code be refactored. We present a framework for applying binomial checkpointing to arbitrary code with no special annotation or refactoring required. This is accomplished by applying binomial checkpointing directly to a program trace. This trace is produced by a general-purpose checkpointing mechanism that is orthogonal to AD.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/22/2017

Divide-and-Conquer Checkpointing for Arbitrary Programs with No User Annotation

Classical reverse-mode automatic differentiation (AD) imposes only a sma...
12/28/2022

Reverse-Mode Automatic Differentiation of Compiled Programs

Tools for algorithmic differentiation (AD) provide accurate derivatives ...
02/21/2022

AD for an Array Language with Nested Parallelism

We present a technique for applying (forward and) reverse-mode automatic...
04/22/2022

You Only Linearize Once: Tangents Transpose to Gradients

Automatic differentiation (AD) is conventionally understood as a family ...
04/09/2020

Automatic Differentiation in ROOT

In mathematics and computer algebra, automatic differentiation (AD) is a...
10/16/2022

Automatic Differentiation of Programs with Discrete Randomness

Automatic differentiation (AD), a technique for constructing new program...
03/10/2020

Differentiate Everything with a Reversible Domain-Specific Language

Traditional machine instruction level reverse mode automatic differentia...