Automatic Reparameterisation of Probabilistic Programs

by   Maria I. Gorinova, et al.

Probabilistic programming has emerged as a powerful paradigm in statistics, applied science, and machine learning: by decoupling modelling from inference, it promises to allow modellers to directly reason about the processes generating data. However, the performance of inference algorithms can be dramatically affected by the parameterisation used to express a model, requiring users to transform their programs in non-intuitive ways. We argue for automating these transformations, and demonstrate that mechanisms available in recent modeling frameworks can implement non-centring and related reparameterisations. This enables new inference algorithms, and we propose two: a simple approach using interleaved sampling and a novel variational formulation that searches over a continuous space of parameterisations. We show that these approaches enable robust inference across a range of models, and can yield more efficient samplers than the best fixed parameterisation.


Stochastic probabilistic programs

We introduce the notion of a stochastic probabilistic program and presen...

TreeFlow: probabilistic programming and automatic differentiation for phylogenetics

Probabilistic programming frameworks are powerful tools for statistical ...

Translating Recursive Probabilistic Programs to Factor Graph Grammars

It is natural for probabilistic programs to use conditionals to express ...

Composing inference algorithms as program transformations

Probabilistic inference procedures are usually coded painstakingly from ...

Universal Marginaliser for Deep Amortised Inference for Probabilistic Programs

Probabilistic programming languages (PPLs) are powerful modelling tools ...

From Probabilistic Programming to Complexity-based Programming

The paper presents the main characteristics and a preliminary implementa...

Linear Models of Computation and Program Learning

We consider two classes of computations which admit taking linear combin...

Code Repositories


Automatic Reparameterisation of Probabilistic Programs

view repo

Please sign up or login with your details

Forgot password? Click here to reset