Log In Sign Up

On Tractable Computation of Expected Predictions

by   Pasha Khosravi, et al.

Computing expected predictions has many interesting applications in areas such as fairness, handling missing values, and data analysis. Unfortunately, computing expectations of a discriminative model with respect to a probability distribution defined by an arbitrary generative model has been proven to be hard in general. In fact, the task is intractable even for simple models such as logistic regression and a naive Bayes distribution. In this paper, we identify a pair of generative and discriminative models that enables tractable computation of expectations of the latter with respect to the former, as well as moments of any order, in case of regression. Specifically, we consider expressive probabilistic circuits with certain structural constraints that support tractable probabilistic inference. Moreover, we exploit the tractable computation of high-order moments to derive an algorithm to approximate the expectations, for classification scenarios in which exact computations are intractable. We evaluate the effectiveness of our exact and approximate algorithms in handling missing data during prediction time where they prove to be competitive to standard imputation techniques on a variety of datasets. Finally, we illustrate how expected prediction framework can be used to reason about the behaviour of discriminative models.


page 1

page 2

page 3

page 4


What to Expect of Classifiers? Reasoning about Logistic Regression with Missing Features

While discriminative classifiers often yield strong predictive performan...

Tractable Computation of Expected Kernels by Circuits

Computing the expectation of some kernel function is ubiquitous in machi...

Entropy methods for the confidence assessment of probabilistic classification models

Many classification models produce a probability distribution as the out...

On Constraint Definability in Tractable Probabilistic Models

Incorporating constraints is a major concern in probabilistic machine le...

Strudel: Learning Structured-Decomposable Probabilistic Circuits

Probabilistic circuits (PCs) represent a probability distribution as a c...

Joints in Random Forests

Decision Trees (DTs) and Random Forests (RFs) are powerful discriminativ...