Source Identification for Mixtures of Product Distributions

by   Spencer L. Gordon, et al.

We give an algorithm for source identification of a mixture of k product distributions on n bits. This is a fundamental problem in machine learning with many applications. Our algorithm identifies the source parameters of an identifiable mixture, given, as input, approximate values of multilinear moments (derived, for instance, from a sufficiently large sample), using 2^O(k^2) n^O(k) arithmetic operations. Our result is the first explicit bound on the computational complexity of source identification of such mixtures. The running time improves previous results by Feldman, O'Donnell, and Servedio (FOCS 2005) and Chen and Moitra (STOC 2019) that guaranteed only learning the mixture (without parametric identification of the source). Our analysis gives a quantitative version of a qualitative characterization of identifiable sources that is due to Tahmasebi, Motahari, and Maddah-Ali (ISIT 2018).


page 1

page 2

page 3

page 4


Self-Improving Voronoi Construction for a Hidden Mixture of Product Distributions

We propose a self-improving algorithm for computing Voronoi diagrams und...

Identifying Mixtures of Bayesian Network Distributions

A Bayesian Network is a directed acyclic graph (DAG) on a set of n rando...

Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm

We study the problem of learning a mixture model of non-parametric produ...

Learning Mixtures of Product Distributions via Higher Multilinear Moments

Learning mixtures of k binary product distributions is a central problem...

Learning Mixtures of Linear Regressions in Subexponential Time via Fourier Moments

We consider the problem of learning a mixture of linear regressions (MLR...

Linear Time Clustering for High Dimensional Mixtures of Gaussian Clouds

Clustering mixtures of Gaussian distributions is a fundamental and chall...

Hadamard Powers and the Identification of Mixtures of Products

The Hadamard Power of a matrix is the matrix consisting of all Hadamard ...