Estimating the inverse trace using random forests on graphs

05/06/2019 ∙ by Simon Barthelmé, et al. ∙ 0

Some data analysis problems require the computation of (regularised) inverse traces, i.e. quantities of the form (q + )^-1. For large matrices, direct methods are unfeasible and one must resort to approximations, for example using a conjugate gradient solver combined with Girard's trace estimator (also known as Hutchinson's trace estimator). Here we describe an unbiased estimator of the regularized inverse trace, based on Wilson's algorithm, an algorithm that was initially designed to draw uniform spanning trees in graphs. Our method is fast, easy to implement, and scales to very large matrices. Its main drawback is that it is limited to diagonally dominant matrices .



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.


  • [1] Greg W Anderson, Alice Guionnet, and Ofer Zeitouni. An introduction to random matrices, volume 118. Cambridge university press, 2010.
  • [2] L Avena and A Gaudilliere. On some random forests with determinantal roots. arXiv preprint arXiv:1310.1723, 2013.
  • [3] L. Avena and A. Gaudillière. Two Applications of Random Spanning Forests. Journal of Theoretical Probability, July 2017.
  • [4] Luca Avena, Fabienne Castell, Alexandre Gaudillière, and Clothilde Mélot. Random forests and networks analysis. Journal of Statistical Physics, 173(3-4):985–1027, 2018.
  • [5] Haim Avron and Sivan Toledo. Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix. Journal of the ACM, 58(2):1–34, April 2011.
  • [6] Richard Barrett, Michael W Berry, Tony F Chan, James Demmel, June Donato, Jack Dongarra, Victor Eijkhout, Roldan Pozo, Charles Romine, and Henk Van der Vorst. Templates for the solution of linear systems: building blocks for iterative methods, volume 43. Siam, 1994.
  • [7] Fan RK Chung and Linyuan Lu. Complex graphs and networks, volume 107. American mathematical society Providence, 2006.
  • [8] A Girard. A fast ‘monte-carlo cross-validation’procedure for large least squares problems with noisy data. Numerische Mathematik, 56(1):1–23, 1989.
  • [9] Didier Girard. Un algorithme simple et rapide pour la validation croisée généralisée sur des problèmes de grande taille. Technical report, 1987.
  • [10] Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The elements of statistical learning. Springer, 2009.
  • [11] Timothy Hunter, Ahmed El Alaoui, and Alexandre Bayen. Computing the log-determinant of symmetric, diagonally dominant matrices in near-linear time. arXiv preprint arXiv:1408.1693, 2014.
  • [12] Michael F Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Communications in Statistics-Simulation and Computation, 19(2):433–450, 1990.
  • [13] Jonathan A Kelner, Lorenzo Orecchia, Aaron Sidford, and Zeyuan Allen Zhu. A simple, combinatorial algorithm for solving sdd systems in nearly-linear time. In

    Proceedings of the forty-fifth annual ACM symposium on Theory of computing

    , pages 911–920. ACM, 2013.
  • [14] Michael W Mahoney et al. Randomized algorithms for matrices and data. Foundations and Trends® in Machine Learning, 3(2):123–224, 2011.
  • [15] Dianne P O’Leary. The block conjugate gradient algorithm and related methods. Linear algebra and its applications, 29:293–322, 1980.
  • [16] Havard Rue and Leonhard Held. Gaussian Markov random fields: theory and applications. Chapman and Hall/CRC, 2005.
  • [17] John W Ruge and Klaus Stüben. Algebraic multigrid. In Multigrid methods, pages 73–130. SIAM, 1987.
  • [18] Michael L Stein, Jie Chen, Mihai Anitescu, et al. Stochastic approximation of score functions for gaussian processes. The Annals of Applied Statistics, 7(2):1162–1191, 2013.
  • [19] David Bruce Wilson. Generating random spanning trees more quickly than the cover time. In Proceedings of the Twenty-eighth Annual ACM Symposium on the Theory of Computing (STOC), volume 96, pages 296–303. Citeseer, 1996.