Source-to-Source Automatic Differentiation of OpenMP Parallel Loops

11/02/2021
by   Jan Hückelheim, et al.
0

This paper presents our work toward correct and efficient automatic differentiation of OpenMP parallel worksharing loops in forward and reverse mode. Automatic differentiation is a method to obtain gradients of numerical programs, which are crucial in optimization, uncertainty quantification, and machine learning. The computational cost to compute gradients is a common bottleneck in practice. For applications that are parallelized for multicore CPUs or GPUs using OpenMP, one also wishes to compute the gradients in parallel. We propose a framework to reason about the correctness of the generated derivative code, from which we justify our OpenMP extension to the differentiation model. We implement this model in the automatic differentiation tool Tapenade and present test cases that are differentiated following our extended differentiation procedure. Performance of the generated derivative programs in forward and reverse mode is better than sequential, although our reverse mode often scales worse than the input programs.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 13

page 25

02/17/2022

Gradients without Backpropagation

Using backpropagation to compute gradients of objective functions for op...
07/05/2019

Automatic Differentiation for Adjoint Stencil Loops

Stencil loops are a common motif in computations including convolutional...
08/28/2019

Forward-Mode Differentiation of Maxwell's Equations

We present a previously unexplored forward-mode differentiation method f...
04/22/2022

You Only Linearize Once: Tangents Transpose to Gradients

Automatic differentiation (AD) is conventionally understood as a family ...
08/25/2020

Differentiating a Tensor Language

How does one compile derivatives of tensor programs, such that the resul...
03/01/2020

Inverse design of photonic crystals through automatic differentiation

Gradient-based inverse design in photonics has already achieved remarkab...
03/11/2022

GPU Accelerated Automatic Differentiation With Clad

Automatic Differentiation (AD) is instrumental for science and industry....
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.