DeepAI
Log In Sign Up

An Equivalence Principle for the Spectrum of Random Inner-Product Kernel Matrices

05/12/2022
by   Yue M. Lu, et al.
0

We consider random matrices whose entries are obtained by applying a (nonlinear) kernel function to the pairwise inner products between n independent data vectors drawn uniformly from the unit sphere in ℝ^d. Our study of this model is motivated by problems in machine learning, statistics, and signal processing, where such inner-product kernel random matrices and their spectral properties play important roles. Under mild conditions on the kernel function, we establish the weak-limit of the empirical spectral distribution of these matrices when d, n →∞ such that n / d^ℓ→κ∈ (0, ∞), for some fixed ℓ∈ℕ and κ∈ℝ. This generalizes an earlier result of Cheng and Singer (2013), who studied the same model in the linear scaling regime (with ℓ = 1 and n/d →κ). The main insight of our work is a general equivalence principle: the spectrum of the random kernel matrix is asymptotically equivalent to that of a simpler matrix model, constructed as the linear combination of a (shifted) Wishart matrix and an independent matrix drawn from the Gaussian orthogonal ensemble. The aspect ratio of the Wishart matrix and the coefficients of the linear combination are determined by ℓ and by the expansion of the kernel function in the orthogonal Hermite polynomial basis. Consequently, the limiting spectrum of the random kernel matrix can be characterized as the free additive convolution between a Marchenko-Pastur law and a semicircle law.

READ FULL TEXT
10/14/2022

Independence of linear spectral statistics and the point process at the edge of Wigner matrices

In the current paper we consider a Wigner matrix and consider an analyti...
09/25/2019

On the spectral property of kernel-based sensor fusion algorithms of high dimensional data

In this paper, we apply local laws of random matrices and free probabili...
01/21/2018

Limiting Distributions of Spectral Radii for Product of Matrices from the Spherical Ensemble

Consider the product of m independent n× n random matrices from the sphe...
09/15/2019

Inner-product Kernels are Asymptotically Equivalent to Binary Discrete Kernels

This article investigates the eigenspectrum of the inner product-type ke...
05/11/2021

Analysis of One-Hidden-Layer Neural Networks via the Resolvent Method

We compute the asymptotic empirical spectral distribution of a non-linea...
10/05/2021

Random matrices in service of ML footprint: ternary random features with no performance loss

In this article, we investigate the spectral behavior of random features...