Distribution Theoretic Semantics for Non-Smooth Differentiable Programming

With the wide spread of deep learning and gradient descent inspired optimization algorithms, differentiable programming has gained traction. Nowadays it has found applications in many different areas as well, such as scientific computing, robotics, computer graphics and others. One of its notoriously difficult problems consists in interpreting programs that are not differentiable everywhere. In this work we define λ_δ, a core calculus for non-smooth differentiable programs and define its semantics using concepts from distribution theory, a well-established area of functional analysis. We also show how λ_δ presents better equational properties than other existing semantics and use our semantics to reason about a simplified ray tracing algorithm. Further, we relate our semantics to existing differentiable languages by providing translations to and from other existing differentiable semantic models. Finally, we provide a proof-of-concept implementation in PyTorch of the novel constructions in this paper.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/31/2020

Differentiable Programming à la Moreau

The notion of a Moreau envelope is central to the analysis of first-orde...
research
05/13/2022

Differentiable programming: Generalization, characterization and limitations of deep learning

In the past years, deep learning models have been successfully applied i...
research
12/08/2016

Implementing Operational calculus on programming spaces for Differentiable computing

We provide an illustrative implementation of an analytic, infinitely-dif...
research
05/31/2022

Differentiable programming for functional connectomics

Mapping the functional connectome has the potential to uncover key insig...
research
07/17/2019

Differentiable Disentanglement Filter: an Application Agnostic Core Concept Discovery Probe

It has long been speculated that deep neural networks function by discov...
research
12/13/2020

Optimization and Learning With Nonlocal Calculus

Nonlocal models have recently had a major impact in nonlinear continuum ...
research
05/01/2018

Live Functional Programming with Typed Holes

This paper develops a dynamic semantics for incomplete functional progra...

Please sign up or login with your details

Forgot password? Click here to reset