Source Identification for Mixtures of Product Distributions

12/29/2020
by   Spencer L. Gordon, et al.
5

We give an algorithm for source identification of a mixture of k product distributions on n bits. This is a fundamental problem in machine learning with many applications. Our algorithm identifies the source parameters of an identifiable mixture, given, as input, approximate values of multilinear moments (derived, for instance, from a sufficiently large sample), using 2^O(k^2) n^O(k) arithmetic operations. Our result is the first explicit bound on the computational complexity of source identification of such mixtures. The running time improves previous results by Feldman, O'Donnell, and Servedio (FOCS 2005) and Chen and Moitra (STOC 2019) that guaranteed only learning the mixture (without parametric identification of the source). Our analysis gives a quantitative version of a qualitative characterization of identifiable sources that is due to Tahmasebi, Motahari, and Maddah-Ali (ISIT 2018).

READ FULL TEXT

page 1

page 2

page 3

page 4

09/28/2021

Self-Improving Voronoi Construction for a Hidden Mixture of Product Distributions

We propose a self-improving algorithm for computing Voronoi diagrams und...
12/22/2021

Identifying Mixtures of Bayesian Network Distributions

A Bayesian Network is a directed acyclic graph (DAG) on a set of n rando...
04/02/2019

Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm

We study the problem of learning a mixture model of non-parametric produ...
03/17/2018

Learning Mixtures of Product Distributions via Higher Multilinear Moments

Learning mixtures of k binary product distributions is a central problem...
12/16/2019

Learning Mixtures of Linear Regressions in Subexponential Time via Fourier Moments

We consider the problem of learning a mixture of linear regressions (MLR...
12/19/2017

Linear Time Clustering for High Dimensional Mixtures of Gaussian Clouds

Clustering mixtures of Gaussian distributions is a fundamental and chall...
01/27/2021

Hadamard Powers and the Identification of Mixtures of Products

The Hadamard Power of a matrix is the matrix consisting of all Hadamard ...