DeepAI
Log In Sign Up

On Tractable Computation of Expected Predictions

10/05/2019
by   Pasha Khosravi, et al.
15

Computing expected predictions has many interesting applications in areas such as fairness, handling missing values, and data analysis. Unfortunately, computing expectations of a discriminative model with respect to a probability distribution defined by an arbitrary generative model has been proven to be hard in general. In fact, the task is intractable even for simple models such as logistic regression and a naive Bayes distribution. In this paper, we identify a pair of generative and discriminative models that enables tractable computation of expectations of the latter with respect to the former, as well as moments of any order, in case of regression. Specifically, we consider expressive probabilistic circuits with certain structural constraints that support tractable probabilistic inference. Moreover, we exploit the tractable computation of high-order moments to derive an algorithm to approximate the expectations, for classification scenarios in which exact computations are intractable. We evaluate the effectiveness of our exact and approximate algorithms in handling missing data during prediction time where they prove to be competitive to standard imputation techniques on a variety of datasets. Finally, we illustrate how expected prediction framework can be used to reason about the behaviour of discriminative models.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/05/2019

What to Expect of Classifiers? Reasoning about Logistic Regression with Missing Features

While discriminative classifiers often yield strong predictive performan...
02/21/2021

Tractable Computation of Expected Kernels by Circuits

Computing the expectation of some kernel function is ubiquitous in machi...
03/28/2021

Entropy methods for the confidence assessment of probabilistic classification models

Many classification models produce a probability distribution as the out...
01/29/2020

On Constraint Definability in Tractable Probabilistic Models

Incorporating constraints is a major concern in probabilistic machine le...
07/18/2020

Strudel: Learning Structured-Decomposable Probabilistic Circuits

Probabilistic circuits (PCs) represent a probability distribution as a c...
06/25/2020

Joints in Random Forests

Decision Trees (DTs) and Random Forests (RFs) are powerful discriminativ...