A theory explaining the limits and performances of algorithms based on simulated annealing in solving sparse hard inference problems

by   Maria Chiara Angelini, et al.

The planted coloring problem is a prototypical inference problem for which thresholds for Bayes optimal algorithms, like Belief Propagation (BP), can be computed analytically. In this paper, we analyze the limits and performances of the Simulated Annealing (SA), a Monte Carlo-based algorithm that is more general and robust than BP, and thus of broader applicability. We show that SA is sub-optimal in the recovery of the planted solution because it gets attracted by glassy states that, instead, do not influence the BP algorithm. At variance with previous conjectures, we propose an analytic estimation for the SA algorithmic threshold by comparing the spinodal point of the paramagnetic phase and the dynamical critical temperature. This is a fundamental connection between thermodynamical phase transitions and out of equilibrium behavior of Glauber dynamics. We also study an improved version of SA, called replicated SA (RSA), where several weakly coupled replicas are cooled down together. We show numerical evidence that the algorithmic threshold for the RSA coincides with the Bayes optimal one. Finally, we develop an approximated analytical theory explaining the optimal performances of RSA and predicting the location of the transition towards the planted solution in the limit of a very large number of replicas. Our results for RSA support the idea that mismatching the parameters in the prior with respect to those of the generative model may produce an algorithm that is optimal and very robust.


page 1

page 2

page 3

page 4


Mismatching as a tool to enhance algorithmic performances of Monte Carlo methods for the planted clique model

Over-parametrization was a crucial ingredient for recent developments in...

Monte Carlo algorithms are very effective in finding the largest independent set in sparse random graphs

The effectiveness of stochastic algorithms based on Monte Carlo dynamics...

Detectability thresholds and optimal algorithms for community structure in dynamic networks

We study the fundamental limits on learning latent community structure i...

Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference

Gradient-descent-based algorithms and their stochastic versions have wid...

An improved Belief Propagation algorithm finds many Bethe states in the random field Ising model on random graphs

We first present an empirical study of the Belief Propagation (BP) algor...

Statistical mechanical analysis of sparse linear regression as a variable selection problem

An algorithmic limit of compressed sensing or related variable-selection...

Cascade of Phase Transitions for Multi-Scale Clustering

We present a novel framework exploiting the cascade of phase transitions...

Please sign up or login with your details

Forgot password? Click here to reset