I-a General relationship:
Let be a full rank matrix, be the absolute determinant of , and and be smooth distributions over , where has full support. The inverse determinant can be related to an expectation over functions of matrix-vector products:
I-B Refined relationship:
Set , where
is the probability density atof a Gaussian with mean 0 and identity covariance. Additionally, let be an expectation over vectors drawn uniformly on the surface of an -dimensional radius one hypersphere, and be an expectation over a chi distribution with degrees of freedom. The following more refined relationship then follows:
Because , Equation 2
also provides an unbiased stochastic estimator for the determinant of a matrix, in terms of matrix-vector products with its inverse:
Experimental validation of this relationship is presented in Figure 1.
Ii Related Work
, but appears not to have been previously published as an explicit identity. Related expressions appear in work on ratios of moments of quadratic forms, and in techniques for rewriting certain determinants in terms of integrals which can be evaluated by Monte Carlo [13, 5, 4]. In the special case of positive symmetric definite , Gaussian quadrature techniques have been used to stochastically estimate determinants . Other work derives stochastic estimators of classes of log determinants [7, 12, 2]. Hadamard’s inequality can be reinterpreted as a stochastic upper bound on in terms of the norms of row or column vectors .
We hope that the stochastic estimators presented in this note will enable new Monte Carlo techniques for estimating, or stochastically bounding, functions of matrix determinants. These relationships may be especially useful in machine learning for training and evaluating both normalizing flow models[3, 10, 8] and Gaussian process kernels .
Thank you to Alex Alemi, Anudhyan Boral, Ricky Chen, Arnaud Doucet, Guy Gur-Ari, Albin Jones, Abhishek Kumar, Peyman Milanfar, Jeffrey Pennington, Christian Szegedy, Srinivas Vasudevan, and Max Vladymyrov for helpful discussion and links to related work.
-  (1996) Some large-scale matrix computation problems. Journal of Computational and Applied Mathematics 74 (1-2), pp. 71–89. Cited by: §II.
-  (2019) Residual flows for invertible generative modeling. In Advances in Neural Information Processing Systems, pp. 9913–9923. Cited by: §II.
-  (2016) Density estimation using real nvp. arXiv preprint arXiv:1605.08803. Cited by: §III.
-  (2018) Stochastic methods for the fermion determinant in lattice quantum chromodynamics. Ph.D. Thesis, Universität Wuppertal, Fakultät für Mathematik und Naturwissenschaften …. Cited by: §II.
-  (1980) A proposal for monte carlo simulations of fermionic systems. Nucl. Phys. B 180 (CERN-TH-2960), pp. 369–377. Cited by: §II.
-  (1893) Resolution d’une question relative aux determinants. Bull. des sciences math. 2, pp. 240–246. Cited by: §II.
-  (2015) Large-scale log-determinant computation through stochastic chebyshev expansions. In International Conference on Machine Learning, pp. 908–917. Cited by: §II.
-  (2019) Invertible convolutional flow. In Advances in Neural Information Processing Systems, pp. 5636–5646. Cited by: §III.
-  (2003) Gaussian processes in machine learning. In Summer School on Machine Learning, pp. 63–71. Cited by: §III.
-  (2015) Variational inference with normalizing flows. arXiv preprint arXiv:1505.05770. Cited by: §III.
-  (2009) Identities for negative moments of quadratic forms in normal variables. Statistics & probability letters 79 (8), pp. 1004–1007. Cited by: §II.
-  (2017) Randomized matrix-free trace and log-determinant estimators. Numerische Mathematik 137 (2), pp. 353–395. Cited by: §II.
-  (1981) Monte carlo integration for lattice gauge theories with fermions. Physics Letters B 99 (4), pp. 333–338. Cited by: §II.
-  (1954) High-temperature equation of state by a perturbation method. i. nonpolar gases. The Journal of Chemical Physics 22 (8), pp. 1420–1426. Cited by: §II.