Joint Probability Estimation Using Tensor Decomposition and Dictionaries

03/03/2022
by   Shaan ul Haque, et al.
0

In this work, we study non-parametric estimation of joint probabilities of a given set of discrete and continuous random variables from their (empirically estimated) 2D marginals, under the assumption that the joint probability could be decomposed and approximated by a mixture of product densities/mass functions. The problem of estimating the joint probability density function (PDF) using semi-parametric techniques such as Gaussian Mixture Models (GMMs) is widely studied. However such techniques yield poor results when the underlying densities are mixtures of various other families of distributions such as Laplacian or generalized Gaussian, uniform, Cauchy, etc. Further, GMMs are not the best choice to estimate joint distributions which are hybrid in nature, i.e., some random variables are discrete while others are continuous. We present a novel approach for estimating the PDF using ideas from dictionary representations in signal processing coupled with low rank tensor decompositions. To the best our knowledge, this is the first work on estimating joint PDFs employing dictionaries alongside tensor decompositions. We create a dictionary of various families of distributions by inspecting the data, and use it to approximate each decomposed factor of the product in the mixture. Our approach can naturally handle hybrid N-dimensional distributions. We test our approach on a variety of synthetic and real datasets to demonstrate its effectiveness in terms of better classification rates and lower error rates, when compared to state of the art estimators.

READ FULL TEXT

page 1

page 5

research
04/18/2023

Estimating Joint Probability Distribution With Low-Rank Tensor Decomposition, Radon Transforms and Dictionaries

In this paper, we describe a method for estimating the joint probability...
research
04/02/2019

Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm

We study the problem of learning a mixture model of non-parametric produ...
research
03/22/2021

Recovery of Joint Probability Distribution from one-way marginals: Low rank Tensors and Random Projections

Joint probability mass function (PMF) estimation is a fundamental machin...
research
12/01/2017

Tensors, Learning, and 'Kolmogorov Extension' for Finite-alphabet Random Vectors

Estimating the joint probability mass function (PMF) of a set of random ...
research
01/30/2021

Estimating the Unique Information of Continuous Variables

Partial information decompositions (PIDs) identify different modes in wh...
research
06/30/2020

Recovering Joint Probability of Discrete Random Variables from Pairwise Marginals

Learning the joint probability of random variables (RVs) lies at the hea...
research
11/22/2019

Visual Relationship Detection with Low Rank Non-Negative Tensor Decomposition

We address the problem of Visual Relationship Detection (VRD) which aims...

Please sign up or login with your details

Forgot password? Click here to reset