DeepAI AI Chat
Log In Sign Up

Differentiate Everything with a Reversible Domain-Specific Language

by   Jin-Guo Liu, et al.

Traditional machine instruction level reverse mode automatic differentiation (AD) faces the problem of having a space overhead that linear to time in order to trace back the computational state, which is also the source of bad time performance. In reversible programming, a program can be executed bi-directionally, which means we do not need extra design to trace back the computational state. This paper answers the question that how practical it is to implement a machine instruction level reverse mode AD in a reversible programming language. By implementing sparse matrix operations and some machine learning applications in our reversible eDSL NiLang, and benchmark the performance with state-of-the-art AD frameworks, our answer is a clear positive. NiLang is an open source r-Turing complete reversible eDSL in Julia. It empowers users the flexibility to tradeoff time, space, and energy rather than caching data into a global tape. Manageable memory allocation makes it a good tool to differentiate GPU kernels too. In this paper, we will also discuss the challenges that we face towards energy efficient, rounding error free reversible computing, mainly from the instruction and hardware perspective.


Differentiate Everything with a Reversible Programming Language

This paper considers the source-to-source automatic differentiation (AD)...

An Exact Bitwise Reversible Integrator

At a fundamental level most physical equations are time reversible. In t...

Semantics for a Turing-complete Reversible Programming Language with Inductive Types

This paper is concerned with the expressivity and denotational semantics...

Verifying a Minimalist Reverse-Mode AD Library

By exploiting a number of relatively subtle programming language feature...

Tricks from Deep Learning

The deep learning community has devised a diverse set of methods to make...

Binomial Checkpointing for Arbitrary Programs with No User Annotation

Heretofore, automatic checkpointing at procedure-call boundaries, to red...

Optimized Sparse Matrix Operations for Reverse Mode Automatic Differentiation

Sparse matrix representations are ubiquitous in computational science an...

Code Repositories


A differential eDSL that can run faster than light and go back to the past.

view repo