DeepAI AI Chat
Log In Sign Up

Getting to the Point. Index Sets and Parallelism-Preserving Autodiff for Pointful Array Programming

by   Adam Paszke, et al.

We present a novel programming language design that attempts to combine the clarity and safety of high-level functional languages with the efficiency and parallelism of low-level numerical languages. We treat arrays as eagerly-memoized functions on typed index sets, allowing abstract function manipulations, such as currying, to work on arrays. In contrast to composing primitive bulk-array operations, we argue for an explicit nested indexing style that mirrors application of functions to arguments. We also introduce a fine-grained typed effects system which affords concise and automatically-parallelized in-place updates. Specifically, an associative accumulation effect allows reverse-mode automatic differentiation of in-place updates in a way that preserves parallelism. Empirically, we benchmark against the Futhark array programming language, and demonstrate that aggressive inlining and type-driven compilation allows array programs to be written in an expressive, "pointful" style with little performance penalty.


A Term-Rewriting Semantics for Imperative Style Programming

Term rewriting systems have a simple syntax and semantics and facilitate...

Introduction to Rank-polymorphic Programming in Remora (Draft)

Remora is a higher-order, rank-polymorphic array-processing programming ...

A Lambda Calculus for Transfinite Arrays: Unifying Arrays and Streams

Array programming languages allow for concise and generic formulations o...

Performance optimization and modeling of fine-grained irregular communication in UPC

The UPC programming language offers parallelism via logically partitione...

Jeopardy: An Invertible Functional Programming Language

Algorithms are ways of mapping problems to solutions. An algorithm is in...

Hailstorm : A Statically-Typed, Purely Functional Language for IoT Applications

With the growing ubiquity of Internet of Things(IoT), more complex logic...

AD for an Array Language with Nested Parallelism

We present a technique for applying (forward and) reverse-mode automatic...