Compiling Universal Probabilistic Programming Languages with Efficient Parallel Sequential Monte Carlo Inference

12/01/2021
by   Daniel Lundén, et al.
0

Probabilistic programming languages (PPLs) allow users to encode arbitrary inference problems, and PPL implementations provide general-purpose automatic inference for these problems. However, constructing inference implementations that are efficient enough is challenging for many real-world problems. Often, this is due to PPLs not fully exploiting available parallelization and optimization opportunities. For example, handling probabilistic checkpoints in PPLs through continuation-passing style transformations or non-preemptive multitasking – as is done in many popular PPLs – often disallows compilation to low-level languages required for high-performance platforms such as GPUs. To solve the checkpoint problem, we introduce the concept of PPL control-flow graphs (PCFGs) – a simple and efficient approach to checkpoints in low-level languages. We use this approach to implement RootPPL: a low-level PPL built on CUDA and C++ with OpenMP, providing highly efficient and massively parallel SMC inference. We also introduce a general method of compiling universal high-level PPLs to PCFGs and illustrate its application when compiling Miking CorePPL – a high-level universal PPL – to RootPPL. The approach is the first to compile a universal PPL to GPUs with SMC inference. We evaluate RootPPL and the CorePPL compiler through a set of real-world experiments in the domains of phylogenetics and epidemiology, demonstrating up to 6x speedups over state-of-the-art PPLs implementing SMC inference.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

07/03/2015

A New Approach to Probabilistic Programming Inference

We introduce and demonstrate a new approach to inference in expressive p...
05/17/2018

Efficient compilation of array probabilistic programs

Probabilistic programming languages are valuable because they allow us t...
10/16/2020

Probabilistic Programming with CuPPL

Probabilistic Programming Languages (PPLs) are a powerful tool in machin...
01/24/2019

TigerGraph: A Native MPP Graph Database

We present TigerGraph, a graph database system built from the ground up ...
05/13/2017

Learning Semantic Correspondences in Technical Documentation

We consider the problem of translating high-level textual descriptions t...
10/05/2020

Wasm/k: Delimited Continuations for WebAssembly

WebAssembly is designed to be an alternative to JavaScript that is a saf...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.