DeepAI AI Chat
Log In Sign Up

CHAD: Combinatory Homomorphic Automatic Differentiation

by   Matthijs Vákár, et al.

We introduce Combinatory Homomorphic Automatic Differentiation (CHAD), a principled, pure, provably correct method for performing forward- and reverse-mode automatic differentiation (AD) on programming languages with expressive features. It implements AD as a compositional, type-respecting source-code transformation that generates purely functional code. This code transformation is principled in the sense that it is the unique homomorphic (structure preserving) extension to expressive languages of the well-known and unambiguous definitions of automatic differentiation for a first-order functional language. Correctness of the method follows by a (compositional) logical relations argument that shows that the semantics of the syntactic derivative is the usual calculus derivative of the semantics of the original program. In their most elegant formulation, the transformations generate code with linear types. However, the transformations can be implemented in a standard functional language without sacrificing correctness. This implementation can be achieved by making use of abstract data types to represent the required linear types, e.g. through the use of a basic module system. In this paper, we detail the method when applied to a simple higher-order language for manipulating statically sized arrays. However, we explain how the methodology applies, more generally, to functional languages with other expressive features. Finally, we discuss how the scope of CHAD extends beyond applications in automatic differentiation to other dynamic program analyses that accumulate data in a commutative monoid.


page 1

page 2

page 3

page 4


Reverse AD at Higher Types: Pure, Principled and Denotationally Correct

We show how to define source-code transformations for forward- and rever...

CHAD for Expressive Total Languages

We show how to apply forward and reverse mode Combinatory Homomorphic Au...

Backpropagation in the Simply Typed Lambda-calculus with Linear Negation

Backpropagation is a classic automatic differentiation algorithm computi...

Automatic Differentiation for ML-family languages: correctness via logical relations

We give a simple, direct and reusable logical relations technique for la...

A language for evaluating derivatives of functionals using automatic differentiation

We present a simple functional programming language, called Dual PCF, th...

Verification of High-Level Transformations with Inductive Refinement Types

High-level transformation languages like Rascal include expressive featu...

Denotational Correctness of Foward-Mode Automatic Differentiation for Iteration and Recursion

We present semantic correctness proofs of forward-mode Automatic Differe...

Code Repositories