DeepAI AI Chat
Log In Sign Up

Automatic Differentiation for ML-family languages: correctness via logical relations

by   Fernando Lucatelli Nunes, et al.

We give a simple, direct and reusable logical relations technique for languages with recursive features and partially defined differentiable functions. We do so by working out the case of Automatic Differentiation (AD) correctness: namely, we present a proof of the dual numbers style AD macro correctness for realistic functional languages in the ML-family. We also show how this macro provides us with correct forward- and reverse-mode AD. The starting point was to interpret a functional programming language in a suitable freely generated categorical structure. In this setting, by the universal property of the syntactic categorical structure, the dual numbers AD macro and the basic ω-cpo-semantics arise as structure preserving functors. The proof follows, then, by a novel logical relations argument. The key to much of our contribution is a powerful monadic logical relations technique for term recursion and recursive types. It provides us with a semantic correctness proof based on a simple approach for denotational semantics, making use only of the very basic concrete model of ω-cpos.


page 1

page 2

page 3

page 4


Logical Relations for Partial Features and Automatic Differentiation Correctness

We present a simple technique for semantic, open logical relations argum...

A language for evaluating derivatives of functionals using automatic differentiation

We present a simple functional programming language, called Dual PCF, th...

Denotational Correctness of Foward-Mode Automatic Differentiation for Iteration and Recursion

We present semantic correctness proofs of forward-mode Automatic Differe...

Correctness of Automatic Differentiation via Diffeologies and Categorical Gluing

We present semantic correctness proofs of Automatic Differentiation (AD)...

CHAD for Expressive Total Languages

We show how to apply forward and reverse mode Combinatory Homomorphic Au...

CHAD: Combinatory Homomorphic Automatic Differentiation

We introduce Combinatory Homomorphic Automatic Differentiation (CHAD), a...

Reverse AD at Higher Types: Pure, Principled and Denotationally Correct

We show how to define source-code transformations for forward- and rever...