CHAD: Combinatory Homomorphic Automatic Differentiation

03/29/2021
by   Matthijs Vákár, et al.
0

We introduce Combinatory Homomorphic Automatic Differentiation (CHAD), a principled, pure, provably correct method for performing forward- and reverse-mode automatic differentiation (AD) on programming languages with expressive features. It implements AD as a compositional, type-respecting source-code transformation that generates purely functional code. This code transformation is principled in the sense that it is the unique homomorphic (structure preserving) extension to expressive languages of the well-known and unambiguous definitions of automatic differentiation for a first-order functional language. Correctness of the method follows by a (compositional) logical relations argument that shows that the semantics of the syntactic derivative is the usual calculus derivative of the semantics of the original program. In their most elegant formulation, the transformations generate code with linear types. However, the transformations can be implemented in a standard functional language without sacrificing correctness. This implementation can be achieved by making use of abstract data types to represent the required linear types, e.g. through the use of a basic module system. In this paper, we detail the method when applied to a simple higher-order language for manipulating statically sized arrays. However, we explain how the methodology applies, more generally, to functional languages with other expressive features. Finally, we discuss how the scope of CHAD extends beyond applications in automatic differentiation to other dynamic program analyses that accumulate data in a commutative monoid.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/10/2020

Reverse AD at Higher Types: Pure, Principled and Denotationally Correct

We show how to define source-code transformations for forward- and rever...
research
10/01/2021

CHAD for Expressive Total Languages

We show how to apply forward and reverse mode Combinatory Homomorphic Au...
research
09/27/2019

Backpropagation in the Simply Typed Lambda-calculus with Linear Negation

Backpropagation is a classic automatic differentiation algorithm computi...
research
10/14/2022

Automatic Differentiation for ML-family languages: correctness via logical relations

We give a simple, direct and reusable logical relations technique for la...
research
10/12/2022

A language for evaluating derivatives of functionals using automatic differentiation

We present a simple functional programming language, called Dual PCF, th...
research
09/17/2018

Verification of High-Level Transformations with Inductive Refinement Types

High-level transformation languages like Rascal include expressive featu...
research
07/10/2020

Denotational Correctness of Foward-Mode Automatic Differentiation for Iteration and Recursion

We present semantic correctness proofs of forward-mode Automatic Differe...

Please sign up or login with your details

Forgot password? Click here to reset