
Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo
Deep Gaussian Processes (DGPs) are hierarchical generalizations of Gauss...
read it

Variational Fourier features for Gaussian processes
This work brings together two powerful concepts in Gaussian processes: t...
read it

Randomly Projected Additive Gaussian Processes for Regression
Gaussian processes (GPs) provide flexible distributions over functions, ...
read it

On Learning High Dimensional Structured Single Index Models
Single Index Models (SIMs) are simple yet flexible semiparametric model...
read it

Randomized GramSchmidt process with application to GMRES
A randomized GramSchmidt algorithm is developed for orthonormalization ...
read it

Nonseparable Gaussian Stochastic Process: A Unified View and Computational Strategy
Gaussian stochastic process (GaSP) has been widely used as a prior over ...
read it

Fast nonlinear embeddings via structured matrices
We present a new paradigm for speeding up randomized computations of sev...
read it
HighDimensional Gaussian Process Inference with Derivatives
Although it is widely known that Gaussian processes can be conditioned on observations of the gradient, this functionality is of limited use due to the prohibitive computational cost of 𝒪(N^3 D^3) in data points N and dimension D. The dilemma of gradient observations is that a single one of them comes at the same cost as D independent function evaluations, so the latter are often preferred. Careful scrutiny reveals, however, that derivative observations give rise to highly structured kernel Gram matrices for very general classes of kernels (inter alia, stationary kernels). We show that in the lowdata regime N<D, the Gram matrix can be decomposed in a manner that reduces the cost of inference to 𝒪(N^2D + (N^2)^3) (i.e., linear in the number of dimensions) and, in special cases, to 𝒪(N^2D + N^3). This reduction in complexity opens up new usecases for inference with gradients especially in the highdimensional regime, where the informationtocost ratio of gradient observations significantly increases. We demonstrate this potential in a variety of tasks relevant for machine learning, such as optimization and Hamiltonian Monte Carlo with predictive gradients.
READ FULL TEXT
Comments
There are no comments yet.