Variance Reduction for Matrix Computations with Applications to Gaussian Processes

06/28/2021
by   Anant Mathur, et al.
0

In addition to recent developments in computing speed and memory, methodological advances have contributed to significant gains in the performance of stochastic simulation. In this paper, we focus on variance reduction for matrix computations via matrix factorization. We provide insights into existing variance reduction methods for estimating the entries of large matrices. Popular methods do not exploit the reduction in variance that is possible when the matrix is factorized. We show how computing the square root factorization of the matrix can achieve in some important cases arbitrarily better stochastic performance. In addition, we propose a factorized estimator for the trace of a product of matrices and numerically demonstrate that the estimator can be up to 1,000 times more efficient on certain problems of estimating the log-likelihood of a Gaussian process. Additionally, we provide a new estimator of the log-determinant of a positive semi-definite matrix where the log-determinant is treated as a normalizing constant of a probability density.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/09/2017

Scalable Log Determinants for Gaussian Process Kernel Learning

For applications as varied as Bayesian neural networks, determinantal po...
research
08/09/2014

A Bayesian Probability Calculus for Density Matrices

One of the main concepts in quantum physics is a density matrix, which i...
research
05/19/2017

A Unified Framework for Stochastic Matrix Factorization via Variance Reduction

We propose a unified framework to speed up the existing stochastic matri...
research
11/04/2014

Expectation-Maximization for Learning Determinantal Point Processes

A determinantal point process (DPP) is a probabilistic model of set dive...
research
06/06/2018

Identifying Heritable Communities of Microbiome by Root-Unifrac and Wishart Distribution

We introduce a method to identify heritable microbiome communities when ...
research
04/02/2018

Asymptotic normality and analysis of variance of log-likelihood ratios in spiked random matrix models

The present manuscript studies signal detection by likelihood ratio test...
research
04/29/2023

Representing Additive Gaussian Processes by Sparse Matrices

Among generalized additive models, additive Matérn Gaussian Processes (G...

Please sign up or login with your details

Forgot password? Click here to reset