Replica theory for learning curves for Gaussian processes on random graphs

02/27/2012
by   Matthew J. Urry, et al.
0

Statistical physics approaches can be used to derive accurate predictions for the performance of inference methods learning from potentially noisy data, as quantified by the learning curve defined as the average error versus number of training examples. We analyse a challenging problem in the area of non-parametric inference where an effectively infinite number of parameters has to be learned, specifically Gaussian process regression. When the inputs are vertices on a random graph and the outputs noisy function values, we show that replica techniques can be used to obtain exact performance predictions in the limit of large graphs. The covariance of the Gaussian process prior is defined by a random walk kernel, the discrete analogue of squared exponential kernels on continuous spaces. Conventionally this kernel is normalised only globally, so that the prior variance can differ between vertices; as a more principled alternative we consider local normalisation, where the prior variance is uniform.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/06/2012

Random walk kernels and learning curves for Gaussian process regression on random graphs

We consider learning on graphs, guided by kernels that encode similarity...
research
11/18/2020

Approximate inference in related multi-output Gaussian Process Regression

In Gaussian Processes a multi-output kernel is a covariance function ove...
research
03/14/2022

On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernel

Deep Gaussian Process as a Bayesian learning model is promising because ...
research
11/02/2012

Learning curves for multi-task Gaussian process regression

We study the average case performance of multi-task Gaussian process (GP...
research
08/31/2022

Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces I: the Compact Case

Gaussian processes are arguably the most important model class in spatia...
research
01/30/2023

Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces II: non-compact symmetric spaces

Gaussian processes are arguably the most important class of spatiotempor...
research
11/03/2022

Isotropic Gaussian Processes on Finite Spaces of Graphs

We propose a principled way to define Gaussian process priors on various...

Please sign up or login with your details

Forgot password? Click here to reset