Getting to the Point. Index Sets and Parallelism-Preserving Autodiff for Pointful Array Programming

by   Adam Paszke, et al.

We present a novel programming language design that attempts to combine the clarity and safety of high-level functional languages with the efficiency and parallelism of low-level numerical languages. We treat arrays as eagerly-memoized functions on typed index sets, allowing abstract function manipulations, such as currying, to work on arrays. In contrast to composing primitive bulk-array operations, we argue for an explicit nested indexing style that mirrors application of functions to arguments. We also introduce a fine-grained typed effects system which affords concise and automatically-parallelized in-place updates. Specifically, an associative accumulation effect allows reverse-mode automatic differentiation of in-place updates in a way that preserves parallelism. Empirically, we benchmark against the Futhark array programming language, and demonstrate that aggressive inlining and type-driven compilation allows array programs to be written in an expressive, "pointful" style with little performance penalty.



There are no comments yet.


page 23


A Term-Rewriting Semantics for Imperative Style Programming

Term rewriting systems have a simple syntax and semantics and facilitate...

Introduction to Rank-polymorphic Programming in Remora (Draft)

Remora is a higher-order, rank-polymorphic array-processing programming ...

A Lambda Calculus for Transfinite Arrays: Unifying Arrays and Streams

Array programming languages allow for concise and generic formulations o...

Performance optimization and modeling of fine-grained irregular communication in UPC

The UPC programming language offers parallelism via logically partitione...

Reference Capabilities for Safe Parallel Array Programming

The array is a fundamental data structure that provides an efficient way...

TRIOT: Faster tensor manipulation in C++11

[abridged] Context: Multidimensional arrays are used by many different a...

Extended abstract: Type oriented programming for task based parallelism

Writing parallel codes is difficult and exhibits a fundamental trade-off...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.