Recovery of Joint Probability Distribution from one-way marginals: Low rank Tensors and Random Projections

by   Jian Vora, et al.

Joint probability mass function (PMF) estimation is a fundamental machine learning problem. The number of free parameters scales exponentially with respect to the number of random variables. Hence, most work on nonparametric PMF estimation is based on some structural assumptions such as clique factorization adopted by probabilistic graphical models, imposition of low rank on the joint probability tensor and reconstruction from 3-way or 2-way marginals, etc. In the present work, we link random projections of data to the problem of PMF estimation using ideas from tomography. We integrate this idea with the idea of low-rank tensor decomposition to show that we can estimate the joint density from just one-way marginals in a transformed space. We provide a novel algorithm for recovering factors of the tensor from one-way marginals, test it across a variety of synthetic and real-world datasets, and also perform MAP inference on the estimated model for classification.



There are no comments yet.


page 1

page 2

page 3

page 4


Completing a joint PMF from projections: a low-rank coupled tensor factorization approach

There has recently been considerable interest in completing a low-rank m...

Recovering Joint Probability of Discrete Random Variables from Pairwise Marginals

Learning the joint probability of random variables (RVs) lies at the hea...

Joint Probability Estimation Using Tensor Decomposition and Dictionaries

In this work, we study non-parametric estimation of joint probabilities ...

Estimation of low-rank tensors via convex optimization

In this paper, we propose three approaches for the estimation of the Tuc...

Tensors, Learning, and 'Kolmogorov Extension' for Finite-alphabet Random Vectors

Estimating the joint probability mass function (PMF) of a set of random ...

Model-Free State Estimation Using Low-Rank Canonical Polyadic Decomposition

As electric grids experience high penetration levels of renewable genera...

Information-theoretic Feature Selection via Tensor Decomposition and Submodularity

Feature selection by maximizing high-order mutual information between th...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.