DeepAI AI Chat
Log In Sign Up

Information Theoretic Bound on Optimal Worst-case Error in Binary Mixture Identification

by   Khashayar Gatmiry, et al.
Sharif Accelerator

Identification of latent binary sequences from a pool of noisy observations has a wide range of applications in both statistical learning and population genetics. Each observed sequence is the result of passing one of the latent mother-sequences through a binary symmetric channel, which makes this configuration analogous to a special case of Bernoulli Mixture Models. This paper aims to attain an asymptotically tight upper-bound on the error of Maximum Likelihood mixture identification in such problems. The obtained results demonstrate fundamental guarantees on the inference accuracy of the optimal estimator. To this end, we set out to find the closest pair of discrete distributions with respect to the Chernoff Information measure. We provide a novel technique to lower bound the Chernoff Information in an efficient way. We also show that a drastic phase transition occurs at noise level 0.25. Our findings reveal that the identification problem becomes much harder as the noise probability exceeds this threshold.


page 1

page 2

page 3

page 4


Information Theoretic Bounds on Optimal Worst-case Error in Binary Mixture Identification

Identification of latent binary sequences from a pool of noisy observati...

Improving Supervised Phase Identification Through the Theory of Information Losses

This paper considers the problem of Phase Identification in power distri...

Maximum Likelihood Upper Bounds on the Capacities of Discrete Information Stable Channels

Motivated by a greedy approach for generating information stable process...

Mean Estimation in High-Dimensional Binary Markov Gaussian Mixture Models

We consider a high-dimensional mean estimation problem over a binary hid...

On the binary adder channel with complete feedback, with an application to quantitative group testing

We determine the exact value of the optimal symmetric rate point in the ...

The All-or-Nothing Phenomenon in Sparse Linear Regression

We study the problem of recovering a hidden binary k-sparse p-dimensiona...

Phase Transitions in the Pooled Data Problem

In this paper, we study the pooled data problem of identifying the label...