Two equalities expressing the determinant of a matrix in terms of expectations over matrix-vector products

05/13/2020 ∙ by Jascha Sohl-Dickstein, et al. ∙ Google 0

We introduce two equations expressing the inverse determinant of a full rank matrix 𝐀∈ℝ^n × n in terms of expectations over matrix-vector products. The first relationship is |det (𝐀)|^-1 = 𝔼_𝐬∼𝒮^n-1[ ‖𝐀𝐬‖^-n], where expectations are over vectors drawn uniformly on the surface of an n-dimensional radius one hypersphere. The second relationship is |det(𝐀)|^-1 = 𝔼_𝐱∼ q[ p(𝐀𝐱) / q(𝐱)], where p and q are smooth distributions, and q has full support.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Derivation

I-a General relationship:

Let be a full rank matrix, be the absolute determinant of , and and be smooth distributions over , where has full support. The inverse determinant can be related to an expectation over functions of matrix-vector products:

(1)

I-B Refined relationship:

Set , where

is the probability density at

of a Gaussian with mean 0 and identity covariance. Additionally, let be an expectation over vectors drawn uniformly on the surface of an -dimensional radius one hypersphere, and be an expectation over a chi distribution with degrees of freedom. The following more refined relationship then follows:

(2)

Because , Equation 2

also provides an unbiased stochastic estimator for the determinant of a matrix, in terms of matrix-vector products with its inverse:

(3)

Experimental validation of this relationship is presented in Figure 1.

Ii Related Work

The equality in Section I-A has been used in physics [14]

, but appears not to have been previously published as an explicit identity. Related expressions appear in work on ratios of moments of quadratic forms

[11], and in techniques for rewriting certain determinants in terms of integrals which can be evaluated by Monte Carlo [13, 5, 4]. In the special case of positive symmetric definite , Gaussian quadrature techniques have been used to stochastically estimate determinants [1]. Other work derives stochastic estimators of classes of log determinants [7, 12, 2]. Hadamard’s inequality can be reinterpreted as a stochastic upper bound on in terms of the norms of row or column vectors [6].

Iii Discussion

We hope that the stochastic estimators presented in this note will enable new Monte Carlo techniques for estimating, or stochastically bounding, functions of matrix determinants. These relationships may be especially useful in machine learning for training and evaluating both normalizing flow models

[3, 10, 8] and Gaussian process kernels [9].

Fig. 1: An experimental validation of the relationship in Equation 3. We stochastically estimate the determinant of a matrix , by averaging over random unit-norm vectors . Here, is a

matrix, with iid, variance one, Gaussian entries.

Acknowledgments

Thank you to Alex Alemi, Anudhyan Boral, Ricky Chen, Arnaud Doucet, Guy Gur-Ari, Albin Jones, Abhishek Kumar, Peyman Milanfar, Jeffrey Pennington, Christian Szegedy, Srinivas Vasudevan, and Max Vladymyrov for helpful discussion and links to related work.

References

  • [1] Z. Bai, G. Fahey, and G. Golub (1996) Some large-scale matrix computation problems. Journal of Computational and Applied Mathematics 74 (1-2), pp. 71–89. Cited by: §II.
  • [2] T. Q. Chen, J. Behrmann, D. K. Duvenaud, and J. Jacobsen (2019) Residual flows for invertible generative modeling. In Advances in Neural Information Processing Systems, pp. 9913–9923. Cited by: §II.
  • [3] L. Dinh, J. Sohl-Dickstein, and S. Bengio (2016) Density estimation using real nvp. arXiv preprint arXiv:1605.08803. Cited by: §III.
  • [4] J. F. Finkenrath (2018) Stochastic methods for the fermion determinant in lattice quantum chromodynamics. Ph.D. Thesis, Universität Wuppertal, Fakultät für Mathematik und Naturwissenschaften …. Cited by: §II.
  • [5] F. Fucito, G. Parisi, E. Marinari, and C. Rebbi (1980) A proposal for monte carlo simulations of fermionic systems. Nucl. Phys. B 180 (CERN-TH-2960), pp. 369–377. Cited by: §II.
  • [6] J. Hadamard (1893) Resolution d’une question relative aux determinants. Bull. des sciences math. 2, pp. 240–246. Cited by: §II.
  • [7] I. Han, D. Malioutov, and J. Shin (2015) Large-scale log-determinant computation through stochastic chebyshev expansions. In International Conference on Machine Learning, pp. 908–917. Cited by: §II.
  • [8] M. Karami, D. Schuurmans, J. Sohl-Dickstein, L. Dinh, and D. Duckworth (2019) Invertible convolutional flow. In Advances in Neural Information Processing Systems, pp. 5636–5646. Cited by: §III.
  • [9] C. E. Rasmussen (2003) Gaussian processes in machine learning. In Summer School on Machine Learning, pp. 63–71. Cited by: §III.
  • [10] D. J. Rezende and S. Mohamed (2015) Variational inference with normalizing flows. arXiv preprint arXiv:1505.05770. Cited by: §III.
  • [11] A. L. Rukhin (2009) Identities for negative moments of quadratic forms in normal variables. Statistics & probability letters 79 (8), pp. 1004–1007. Cited by: §II.
  • [12] A. K. Saibaba, A. Alexanderian, and I. C. Ipsen (2017) Randomized matrix-free trace and log-determinant estimators. Numerische Mathematik 137 (2), pp. 353–395. Cited by: §II.
  • [13] D. Weingarten and D. Petcher (1981) Monte carlo integration for lattice gauge theories with fermions. Physics Letters B 99 (4), pp. 333–338. Cited by: §II.
  • [14] R. W. Zwanzig (1954) High-temperature equation of state by a perturbation method. i. nonpolar gases. The Journal of Chemical Physics 22 (8), pp. 1420–1426. Cited by: §II.