Conditional independence by typing

by   Maria I. Gorinova, et al.

A central goal of probabilistic programming languages (PPLs) is to separate modelling from inference. However, this goal is hard to achieve in practice. Users are often forced to re-write their models in order to improve efficiency of inference or meet restrictions imposed by the PPL. Conditional independence (CI) relationships among parameters are a crucial aspect of probabilistic models that captures a qualitative summary of the specified model and can facilitate more efficient inference. We present an information flow type system for probabilistic programming that captures conditional independence (CI) relationships, and show that, for a well-typed program in our system, the distribution it implements is guaranteed to have certain CI-relationships. Further, by using type inference, we can statically deduce which CI-properties are present in a specified model. As a practical application, we consider the problem of how to perform inference on models with mixed discrete and continuous parameters. Inference on such models is challenging in many existing PPLs, but can be improved through a workaround, where the discrete parameters are used implicitly, at the expense of manual model re-writing. We present a source-to-source semantics-preserving transformation, which uses our CI-type system to automate this workaround by eliminating the discrete parameters from a probabilistic program. The resulting program can be seen as a hybrid inference algorithm on the original program, where continuous parameters can be drawn using efficient gradient-based inference methods, while the discrete parameters are drawn using variable elimination. We implement our CI-type system and its example application in SlicStan: a compositional variant of Stan.



page 1

page 2

page 3

page 4


Program Analysis of Probabilistic Programs

Probabilistic programming is a growing area that strives to make statist...

Scaling Exact Inference for Discrete Probabilistic Programs

Probabilistic programming languages (PPLs) are an expressive means of re...

Automated learning with a probabilistic programming language: Birch

This work offers a broad perspective on probabilistic modeling and infer...

Universal Marginaliser for Deep Amortised Inference for Probabilistic Programs

Probabilistic programming languages (PPLs) are powerful modelling tools ...

Deep Amortized Inference for Probabilistic Programs

Probabilistic programming languages (PPLs) are a powerful modeling tool,...

Swift: Compiled Inference for Probabilistic Programming Languages

A probabilistic program defines a probability measure over its semantic ...

CatBoostLSS – An extension of CatBoost to probabilistic forecasting

We propose a new framework of CatBoost that predicts the entire conditio...

Code Repositories


A Stan-like probabilistic programming language.

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.