Consistent recovery threshold of hidden nearest neighbor graphs

11/18/2019 ∙ by Jian Ding, et al. ∙ 0

Motivated by applications such as discovering strong ties in social networks and assembling genome subsequences in biology, we study the problem of recovering a hidden 2k-nearest neighbor (NN) graph in an n-vertex complete graph, whose edge weights are independent and distributed according to P_n for edges in the hidden 2k-NN graph and Q_n otherwise. The special case of Bernoulli distributions corresponds to a variant of the Watts-Strogatz small-world graph. We focus on two types of asymptotic recovery guarantees as n→∞: (1) exact recovery: all edges are classified correctly with probability tending to one; (2) almost exact recovery: the expected number of misclassified edges is o(nk). We show that the maximum likelihood estimator achieves (1) exact recovery for 2 < k < n^o(1) if lim inf2α_n/log n>1; (2) almost exact recovery for 1 < k < o( log n/loglog n) if lim infkD(P_n||Q_n)/log n>1, where α_n -2 log∫√(d P_n d Q_n) is the Rényi divergence of order 1/2 and D(P_n||Q_n) is the Kullback-Leibler divergence. Under mild distributional assumptions, these conditions are shown to be information-theoretically necessary for any algorithm to succeed. A key challenge in the analysis is the enumeration of 2k-NN graphs that differ from the hidden one by a given number of edges.



There are no comments yet.


page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.