A Differential-form Pullback Programming Language for Higher-order Reverse-mode Automatic Differentiation

02/19/2020
by   Carol Mak, et al.
0

Building on the observation that reverse-mode automatic differentiation (AD) – a generalisation of backpropagation – can naturally be expressed as pullbacks of differential 1-forms, we design a simple higher-order programming language with a first-class differential operator, and present a reduction strategy which exactly simulates reverse-mode AD. We justify our reduction strategy by interpreting our language in any differential λ-category that satisfies the Hahn-Banach Separation Theorem, and show that the reduction strategy precisely captures reverse-mode AD in a truly higher-order setting.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

05/20/2021

Decomposing reverse-mode automatic differentiation

We decompose reverse-mode automatic differentiation into (forward-mode) ...
11/10/2016

Efficient Implementation of a Higher-Order Language with Built-In AD

We show that Automatic Differentiation (AD) operators can be provided in...
03/27/2018

Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator

Deep learning has seen tremendous success over the past decade in comput...
04/02/2018

The simple essence of automatic differentiation

Automatic differentiation (AD) in reverse mode (RAD) is a central compon...
04/02/2018

The simple essence of automatic differentiation (Differentiable functional programming made easy)

Automatic differentiation (AD) in reverse mode (RAD) is a central compon...
12/30/2018

A Geometric Theory of Higher-Order Automatic Differentiation

First-order automatic differentiation is a ubiquitous tool across statis...
06/27/2021

Automatic Differentiation With Higher Infinitesimals, or Computational Smooth Infinitesimal Analysis in Weil Algebra

We propose an algorithm to compute the C^∞-ring structure of arbitrary W...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.