Higher Order Automatic Differentiation of Higher Order Functions

01/17/2021
by   Mathieu Huot, et al.
0

We present semantic correctness proofs of automatic differentiation (AD). We consider a forward-mode AD method on a higher order language with algebraic data types, and we characterise it as the unique structure preserving macro given a choice of derivatives for basic operations. We describe a rich semantics for differentiable programming, based on diffeological spaces. We show that it interprets our language, and we phrase what it means for the AD method to be correct with respect to this semantics. We show that our characterisation of AD gives rise to an elegant semantic proof of its correctness based on a gluing construction on diffeological spaces. We explain how this is, in essence, a logical relations argument. Throughout, we show how the analysis extends to AD methods for computing higher order derivatives using a Taylor approximation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/07/2020

Correctness of Automatic Differentiation via Diffeologies and Categorical Gluing

We present semantic correctness proofs of Automatic Differentiation (AD)...
research
07/10/2020

Denotational Correctness of Foward-Mode Automatic Differentiation for Iteration and Recursion

We present semantic correctness proofs of forward-mode Automatic Differe...
research
11/30/2021

Towards Denotational Semantics of AD for Higher-Order, Recursive, Probabilistic Languages

Automatic differentiation (AD) aims to compute derivatives of user-defin...
research
06/27/2021

Automatic Differentiation With Higher Infinitesimals, or Computational Smooth Infinitesimal Analysis in Weil Algebra

We propose an algorithm to compute the C^∞-ring structure of arbitrary W...
research
07/15/2020

λ_S: Computable semantics for differentiable programming with higher-order functions and datatypes

Deep learning is moving towards increasingly sophisticated optimization ...
research
11/06/2020

Automatic Differentiation in PCF

We study the correctness of automatic differentiation (AD) in the contex...
research
12/21/2022

Forward- or Reverse-Mode Automatic Differentiation: What's the Difference?

Automatic differentiation (AD) has been a topic of interest for research...

Please sign up or login with your details

Forgot password? Click here to reset