DeepAI AI Chat
Log In Sign Up

An Operator Theoretic Approach to Nonparametric Mixture Models

by   Robert A. Vandermeulen, et al.
University of Michigan

When estimating finite mixture models, it is common to make assumptions on the mixture components, such as parametric assumptions. In this work, we make no distributional assumptions on the mixture components and instead assume that observations from the mixture model are grouped, such that observations in the same group are known to be drawn from the same mixture component. We precisely characterize the number of observations n per group needed for the mixture model to be identifiable, as a function of the number m of mixture components. In addition to our assumption-free analysis, we also study the settings where the mixture components are either linearly independent or jointly irreducible. Furthermore, our analysis considers two kinds of identifiability -- where the mixture model is the simplest one explaining the data, and where it is the only one. As an application of these results, we precisely characterize identifiability of multinomial mixture models. Our analysis relies on an operator-theoretic framework that associates mixture models in the grouped-sample setting with certain infinite-dimensional tensors. Based on this framework, we introduce general spectral algorithms for recovering the mixture components and illustrate their use on a synthetic data set.


page 1

page 2

page 3

page 4


On The Identifiability of Mixture Models from Grouped Samples

Finite mixture models are statistical models which appear in many proble...

Generalized Identifiability Bounds for Mixture Models with Grouped Samples

Recent work has shown that finite mixture models with m components are i...

Evidence estimation in finite and infinite mixture models and applications

Estimating the model evidence - or mariginal likelihood of the data - is...

Consistent Estimation of Identifiable Nonparametric Mixture Models from Grouped Observations

Recent research has established sufficient conditions for finite mixture...

On the Bias of the Score Function of Finite Mixture Models

We characterize the unbiasedness of the score function, viewed as an inf...

Finite mixture models are typically inconsistent for the number of components

Scientists and engineers are often interested in learning the number of ...

Using Multiple Samples to Learn Mixture Models

In the mixture models problem it is assumed that there are K distributio...