The Circuit Complexity of Inference
share
Belief propagation is one of the foundations of probabilistic and causal reasoning. In this paper, we study the circuit complexity of some of the various tasks it performs. Specifically, in the broadcast tree model (which has important applications to phylogenetic reconstruction and close connections to community detection), we show the following: (1) No AC^0 circuit can guess the label of the root with positive advantage over random guessing, independent of the depth for any non-trivial choice of parameters. (2) There is a TC^0 circuit that competes with the Bayes optimal predictor in some range of parameters above the Kesten-Stigum bound. (3) There is a 16 label broadcast tree model in which it is possible to accurately guess the label of the root, but beating random guessing is NC^1-hard. Our work yields a simple and natural generative model where large depth really is necessary for performing various types of inference, that have intriguing parallels with phase transitions from statistical physics.
READ FULL TEXT