Gauss quadrature for matrix inverse forms with applications

12/07/2015
by   Chengtao Li, et al.
0

We present a framework for accelerating a spectrum of machine learning algorithms that require computation of bilinear inverse forms u^ A^-1u, where A is a positive definite matrix and u a given vector. Our framework is built on Gauss-type quadrature and easily scales to large, sparse matrices. Further, it allows retrospective computation of lower and upper bounds on u^ A^-1u, which in turn accelerates several algorithms. We prove that these bounds tighten iteratively and converge at a linear (geometric) rate. To our knowledge, ours is the first work to demonstrate these key properties of Gauss-type quadrature, which is a classical and deeply studied topic. We illustrate empirical consequences of our results by using quadrature to accelerate machine learning tasks involving determinantal point processes and submodular optimization, and observe tremendous speedups in several instances.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/12/2018

Accelerated Stochastic Matrix Inversion: General Theory and Speeding up BFGS Rules for Faster Second-Order Optimization

We present the first accelerated randomized algorithm for solving linear...
research
02/24/2021

Kronecker Products, Low-Depth Circuits, and Matrix Rigidity

For a matrix M and a positive integer r, the rank r rigidity of M is the...
research
06/21/2023

Condition numbers for the Moore-Penrose inverse and the least squares problem involving rank-structured matrices

Perturbation theory plays a crucial role in sensitivity analysis, which ...
research
12/12/2022

A Roadmap to Domain Knowledge Integration in Machine Learning

Many machine learning algorithms have been developed in recent years to ...
research
07/02/2022

A Structured Sparse Neural Network and Its Matrix Calculations Algorithm

Gradient descent optimizations and backpropagation are the most common m...
research
10/11/2019

Random Quadratic Forms with Dependence: Applications to Restricted Isometry and Beyond

Several important families of computational and statistical results in m...

Please sign up or login with your details

Forgot password? Click here to reset