The Random Conditional Distribution for Higher-Order Probabilistic Inference

by   Zenna Tavares, et al.

The need to condition distributional properties such as expectation, variance, and entropy arises in algorithmic fairness, model simplification, robustness and many other areas. At face value however, distributional properties are not random variables, and hence conditioning them is a semantic error and type error in probabilistic programming languages. On the other hand, distributional properties are contingent on other variables in the model, change in value when we observe more information, and hence in a precise sense are random variables too. In order to capture the uncertain over distributional properties, we introduce a probability construct -- the random conditional distribution -- and incorporate it into a probabilistic programming language Omega. A random conditional distribution is a higher-order random variable whose realizations are themselves conditional random variables. In Omega we extend distributional properties of random variables to random conditional distributions, such that for example while the expectation a real valued random variable is a real value, the expectation of a random conditional distribution is a distribution over expectations. As a consequence, it requires minimal syntax to encode inference problems over distributional properties, which so far have evaded treatment within probabilistic programming systems and probabilistic modeling in general. We demonstrate our approach case studies in algorithmic fairness and robustness.



page 1

page 2

page 3

page 4


A Kolmogorov-Smirnov type test for two inter-dependent random variables

Consider n iid random variables, where ξ_1, ..., ξ_n are n realisations ...

Expectation Programming

Building on ideas from probabilistic programming, we introduce the conce...

Compositional Semantics for Probabilistic Programs with Exact Conditioning

We define a probabilistic programming language for Gaussian random varia...

The basic distributional theory for the product of zero mean correlated normal random variables

The product of two zero mean correlated normal random variables has rece...

A General Derivative Identity for the Conditional Expectation with Focus on the Exponential Family

Consider a pair of random vectors (𝐗,𝐘) and the conditional expectation ...

On Least Squares Linear Regression Without Second Moment

If X and Y are real valued random variables such that the first moments ...

Higher Order Kernel Mean Embeddings to Capture Filtrations of Stochastic Processes

Stochastic processes are random variables with values in some space of p...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.