DeepAI AI Chat
Log In Sign Up

Reverse derivative categories

by   Robin Cockett, et al.

The reverse derivative is a fundamental operation in machine learning and automatic differentiation. This paper gives a direct axiomatization of a category with a reverse derivative operation, in a similar style to that given by Cartesian differential categories for a forward derivative. Intriguingly, a category with a reverse derivative also has a forward derivative, but the converse is not true. In fact, we show explicitly what a forward derivative is missing: a reverse derivative is equivalent to a forward derivative with a dagger structure on its subcategory of linear maps. Furthermore, we show that these linear maps form an additively enriched category with dagger biproducts.


page 1

page 2

page 3

page 4


Jacobians and Gradients for Cartesian Differential Categories

Cartesian differential categories come equipped with a differential comb...

Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits

We introduce Reverse Derivative Ascent: a categorical analogue of gradie...

Monoidal Reverse Differential Categories

Cartesian reverse differential categories (CRDCs) are a recently defined...

Reverse Back Propagation to Make Full Use of Derivative

The development of the back-propagation algorithm represents a landmark ...

Categories of Differentiable Polynomial Circuits for Machine Learning

Reverse derivative categories (RDCs) have recently been shown to be a su...

Computer-supported Exploration of a Categorical Axiomatization of Modeloids

A modeloid, a certain set of partial bijections, emerges from the idea t...

Categorical semantics of a simple differential programming language

With the increased interest in machine learning, and deep learning in pa...